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Abstract 



This paper presents the current state of the art on attack and defense modeling approaches 
that are based on directed acyclic graphs (DAGs). DAGs allow for a hierarchical decomposi- 
tion of complex scenarios into simple, easily understandable and quantifiable actions. Methods 
based on threat trees and Bayesian networks are two well-known approaches to security mod- 
eling. However there exist more than 30 DAG-based methodologies, each having different 
features and goals. The objective of this survey is to present a complete overview of graphical 
, attack and defense modeling techniques based on DAGs. This consists of summarizing the 

existing methodologies, comparing their features and proposing a taxonomy of the described 
formalisms. This article also supports the selection of an adequate modeling technique de- 
Qh . pending on user requirements. 

u 

1 Introduction 

Graphical security models provide a useful method to represent and analyze security scenarios 
that examine vulnerabilities of systems and organizations. The great advantage of graph-based 
approaches lies in combining user friendly, intuitive, visual features with formal semantics and 
algorithms that allow for qualitative and quantitative analysis. Over the course of the last two 

^ i | decades, graphical approaches attracted the attention of numerous security and formal methods 

experts and are quickly becoming a stand-alone research area with dedicated national and inter- 
national research projects [14,17,241,263,273]. Graphical models constitute a valuable support 
tool to facilitate threat assessment and risk management of real-life systems. Thus, they have also 
become popular in the industrial sector. Notable application domains of graphical models include 
security analysis of supervisory control and data acquisition (SCADA) systems [43,257,258], vot- 
ing systems [32,142], vehicular communication systems [4,97], Internet related attacks [148,261], 
secure software engineering [115], and socio-technical attacks [19,77,220]. 

In this paper we focus on graphical methods for analysis of attack and defense scenarios. We 
understand attack and defense scenarios in a general sense: they encompass any malicious action 
of an attacker who wants to harm or damage another party or its assets as well as any defense 
or countermeasure that could be used to prevent or mitigate such malicious actions. In 1991, 
Weiss [286] introduced threat logic trees as the first graphical attack modeling technique. The 
obvious similarity of threat logic trees to fault trees [270] suggests that graph-based security 
modeling has its roots in safety modeling. Weiss' approach can be seen as the origin of numerous 
subsequent models, including attack trees [230, 234] which are nowadays one of the most popular 
graphical security models. 

Today, more than 30 different approaches for analysis of attack and defense scenarios exist. 
Most of them extend the original model of threat logic trees in one or several dimensions which 
include defensive components, timed and ordered actions, dynamic aspects and different types of 
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quantification. Moreover, methods for computation of various security related parameters, such as 
the cost, the impact or likelihood of an attack, the efficiency of necessary protection measures, or 
the environmental damage of an attack, have been developed or adapted. 

This survey concentrates on formalisms based on directed acyclic graphs (DAGs), rather than 
on arbitrary graphs. Described approaches can be divided into two main classes: formalisms derived 
from or extending threat trees, and formalisms based on Bayesian networks. The model creation 
in all threat tree-based methodologies starts with the identification of a feared event represented 
as the root node. Then, the event's causes or consequences, depending on the specific approach, 
are deduced and depicted as refining nodes. The refinement process is illustrated in Figure 1, 
which recreates the first threat tree model proposed by Weiss [286] . The DAG structure allows to 
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Figure 1: A threat logic tree taken from [286]: Obtaining administrator privileges on a UNIX 
system. 



use refinements with a customizable level of detail. The root of a DAG is refined as long as the 
refining children provide useful and adequate information about the modeled scenario. Refinements 
paired with the acyclic structure allow for modularization which in turn allows different experts 
to work in parallel on the same model. This is highly appreciated in case of large-scale, complex 
models, where analysis of different parts requires different types of expertise. A big advantage of 
the DAG-based approaches is that they are fairly scalable. They do not suffer from the state space 
explosion problem, which is common for models based on general graphs with cycles. In the case of 
trees, most of the analysis algorithms are linear with respect to the number of nodes of the model. 
Due to multiple incoming edges, this property is no longer true for DAGs and the complexity of 
analysis methods might in theory be exponential. However in practice, the largest exponent in 
the runtime of DAG-based approaches is still acceptable, since it can be kept small due to the 
underlying cycle-free structure. This is, for instance, the case for Bayesian inference algorithms 
used for the analysis of security models based on Bayesian networks. Figure 2 depicts a simple 
Bayesian attack graph borrowed from [213] and illustrates how to compute the unconditional 
probability of a vulnerability exploitation. 

This paper surveys DAG-based graphical formalisms for attack and defense modeling. These 
formalisms provide a systematic, intuitive and practical representation of a large amount of possible 
attacks, vulnerabilities and countermeasures, while at the same time allowing for an efficient 
formal and quantitative analysis of security scenarios. The contribution of this work is to provide 
a complete overview of the field and systematize existing knowledge. More specifically, the survey 

• presents the state of the art in the field of DAG-based graphical attack and defense modeling 
as of 2012; 
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Figure 2: Bayesian attack graph taken from [213]: A test network with local conditional probability 
distributions (tables) and updated unconditional probabilities (below each table). 



• identifies relevant key aspects allowing to compare different formalisms; 

• proposes a taxonomy of the presented approaches, which helps in selecting an appropriate 
formalism; 

• lays a foundation for future research in the field, with the goal to prevent reinvention of 
already existing features. 

In Section 2, we introduce terminology used in the field of graph-based security modeling and 
provide a template for the description of the formalisms. Section 3 is the main part of the survey 
and presents the DAG-based attack and defense modeling approaches published before 2013. In 
Section 4, we provide a concise tabular overview of the presented formalisms. We illustrate how to 
use the tables in order to select the most relevant modeling technique, depending on the application 
requirements. Section 5 briefly mentions alternative graphical security models. We close the survey 
with concluding section, which summarizes our findings and proposes future research directions in 
the field. 



2 Preliminaries 

In this section we introduce our terminology and make a link to existing definitions and concepts. 
We then present and define the aspects that we have taken into account while analyzing differ- 
ent formalisms. We conclude with a detailed description of how formalisms from Section 3 are 
described. 

2.1 Keywords and Terminology 

When examining different models in the same context, it is imperative to have a common language. 
Over the last 20 years, numerous concepts and definitions have emerged in the field of graphical 
security modeling. This section is intended to introduce the language used in this paper, and to 
serve as quick reference guide over the most commonly occurring concepts. Our goal here is not 
to point out the differences in definitions or other intricate details. 
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Attack and defense modeling By techniques for attack and defense modeling we understand 
formalisms that serve for representation and analysis of malicious behavior of an attacker and 
allow to reason about possible defending strategies of the attacker's opponent, called the defender. 
In our survey we use attacks in a very broad sense. Attacks can also be thought of as threats, 
obstacles, and vulnerabilities. Contrary, defenses can appear in form of protections, mitigations, 
responses and countermeasures. They oppose, mitigate or prevent attacks. 

Nodes Nodes, also called vertices, are one of the main components of graph-based security 
models. They are used to depict the concept that is being modeled. Nodes may represent events, 
goals, objectives and actions. Depending on whether the models are constructed in an inductive 
or deductive way, nodes may also express causes or consequences. 

Root node In a rooted DAG (and therefore in any tree) the root is the single designated node 
that does not have any predecessor. From it all other nodes can be reached via a directed path. 
This distinguished node usually depicts the entire concept which is being modeled. In the context 
of security models, various existing names for this special node include top event, main goal, main 
consequence, main objective or main action. 

Leaf nodes In a DAG, nodes that do not have any children are called leaves. They usually 
display an atomic component of a scenario that is no longer refined. They are also called primary 
events, basic components, elementary attacks, elementary components or basic actions. 

Edges Edges are the second main component of graph-based security models. They link nodes 
with each other, and thusly determine relations between the modeled concepts. Edges are also 
called arcs, arrows, or lines. In some models, edges may have special semantics and may detail a 
cause-consequence relation, a specialization or some other information. 

Connectors Connectors usually specify more preciously how a parent node is connected with 
its children. A connector might be a set of edges or a node of a special type. Connectors are also 
called refinements or gates. Some examples include: AND, OR, XOR, fc-out-of-n, priority AND, 
triggers, etc. 

Priority AND A priority AND (PAND) is a special kind of AND connector which prescribes 
an order in which the nodes are to be treated. The origin of the prescribed order is usually time or 
some priority criterion. The PAND is also called an ordered- AND, an O-AND or a sequential AND. 
Sometimes the underlying reason behind the priority is specified as in the case of the time-based 
AND. 

Attributes Attributes represent aspects or properties that are relevant for quantitative analysis 
of security models. Examples of attributes, sometimes also called metrics, include: impact of an 
attack, costs of necessary defenses, risk associated with an attack etc. Proposed computation 
methods range from versatile approaches that can be applied for evaluation of a wide class of 
attributes, to specific algorithms developed for particular measures. An example of the former is 
the formalization of an attribute domain proposed in [161], which is well suited for calculation of 
any attribute whose underlying algebraic structure is a semi-ring. An example of the latter are 
the specific methods for probability computation proposed in [296]. 

2.2 Examined Aspects 

One of the goals of this paper is to provide a classification of existing formalisms for attack and 
defense modeling. Thus, all approaches described in Section 3 were analyzed based on the same 
13 criteria, which we refer to as aspects and define in this section. 

The formalisms are grouped according to the following two main aspects: 
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1. Attack and/or defense modeling: Attack modeling techniques are focused on an at- 
tacker's actions and vulnerabilities of systems; defense modeling techniques concentrate on 
defensive aspects, such as detection, reaction, responses and prevention. 

2. Static or sequential approaches: Sequential formalisms take temporal aspects, such as 
dynamics time variations, and dependencies between considered actions, such as order or 
priority, into account; static approaches cannot model any of such relations. 

The above two aspects provide a partition of all considered approaches. Furthermore, they 
correspond to questions that a user selecting a suitable formalism is most likely to ask, namely 
'What do we want to model?' and 'How do we want to model?'. The proposed classification allows 
a reader to easily make a primary selection and identify which formalisms best fit his needs. 

Besides the two main aspects, each formalism is analyzed according to additional criteria, listed 
in Table 1. All aspects taken into account in our work, can be grouped into three categories: 

• Aspects relating to the formalism's modeling capabilities, i.e., what we can model: attack or 
defense modeling, sequential or static modeling, quantification, main purpose, extensions. 

• Aspects relating to the formalism's characteristics, i.e., how we can model: structure, con- 
nectors, formalization. 

• Aspects related to the formalism's maturity and usability: tool availability, case study, ex- 
ternal use, paper count, year. 

In Table 1, we define all 13 aspects in form of questions and provide possible values that answer 
the questions. 



Table 1: Table summarizing aspects taken into account in formalism description. 



Aspect 


Aspect Description 


Possible 
Values 


Value Explanation 


Attack or 
defense 


Is the formalism 


Attack 


Only attack modeling 


offensively or 


Defense 


Only defense modeling 


defensively oriented? 


Both 


Integrates attack and defense modeling 


Static or 
sequential 


Can the formalism deal 
with dependencies and 


Static 


Does not support any 
dependencies 


time varying scenarios? 


Sequential 


Supports time and order dependencies 






Versatile 


Supports numerous generic and diverse 
metrics 




Can numerical values 




Quantification 


be computed using the 
formalism? 


Specific 


Dedicated, tailored for (a couple of) 
specific metrics 






No 


Does not support quantification 
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Sec. mod. General security modeling 







Unification 


Unification of existing formalisms 






Quantitative 


JT 1 UVIUc Ut:L Lcl lllt;Lll(J(_lb 1(J1 L[l_lcLllLl ld~ 

tive analysis 


Main purpose 


Why was the formalism 
invented? 


XV1SK 


ouppoii iisk assessment 




Soft. dev. 


Support secure software development 






Int. det. 


Automated intrusion detection and re- 






sponse analysis 






Req. eng. 


Support security requirements enginee- 
ring 






Structural 


New connectors, extended graph struc- 
ture 






Computa- 


How the formalism handles computa- 






tional 


f l r\ n o 1 a rf i~r~\r\ r\ Atirn 1 
LlUllo 1 C.c., lL>p UUW1I 1 


Extensions 


What are added 
features of the 


Quantitative 


VV 111C11 COllipUlcLllOllh Celll Uc pt;i l(Ji 111L.U 

(e.g., specific attributes) 


formalism with respect 
to the state of the art? 


Time 


1 no lArm o 1 i om ^ on n o n n m i~i m a r\ a~r\a~r~\ 
J_ lie 1(J1 111 dl loll 1 Cdll IlcLllU.lt; Lllllc (_lcpt;ll- 

dencies 






Order 


The formalism can handle order depen- 
dencies 






New 

formalism 


Entirely new formalism 




Which graphical 
structure is the 


Tree 


Tree (possibly with repeated nodes) 


Structure 


DAG 


Directed acyclic graph 




formalism based on? 


Tlnsnecifier] 


It is not specified whether the models 
are DAGs or trees 


Connectors 


What type of 
connectors does the 
formalism use? 


J_JloL UI 

connectors 


AND, OR, trigger, sequential AND, 

nrH ^t*a H A l\TTl t"\t*i C\\"\ \"\t A T\TT) nnf" nf 
(JILIGIGLL -TV1N.L/, [JLLKJLLLy XA.1\J_-/, l\, (JUL (Jl 

n, OWA nodes, split gate, countermea- 






sures, counter leaves 




Is the formalism 
formally defined? 


Formal 


Defined using a mathematical frame- 
work; with clear syntax and semantics 


Formalization 


Semi-formal 


Parts of the definitions are given 
verbally, parts are precise 






Informal 


Models only verbally described 


Tool 

availability 


Does a software tool 


Commercial 


A commercial software tool exists 


supporting the 


Prototype 


A prototype tool exists 


formalism exist? 


No 


No implementation exists 



G 



Case study 


Do papers or reports 
describing case studies 
exist? 


Real(istic) 


Real or realistic case study has been 
documented 


Toy case 
study 


Toy case study has been described 


No 


l\Tr> nnpninpntpn pjiqp Qtnn\r pviQf 

1MJ LLUL LHlltilLCl_l LdftC o L Lid V CA15L 


External use 


Do papers or reports 
having a disjoint set of 
authors from the 
formalism inventors 
exist? 


Independent 


People and institutions who did not in- 

^rpnt tnp TnrTTin iqtti nflTfA iiqaH it 

VC11L Lilt 1U1 lilcLllolil lldVL LlotLl 1L 


Collaboration 


the formalism has been used by exter- 
nal researchers and institutions in col- 
laboration with its inventors 


No 


The formalism has only been used by 
its inventors or within the institution 
where it was invented 


Paper count 


How many papers on 
the formalism exist? 


Number 


Number of papers that have been iden- 
tified 1 




What year was the 






Year 


formalism first 


Year 


Before 2013 




published? 







2.3 Template of the Formalism Descriptions 

The description of each formalism presented in Section 3 complies with the following template. 

General presentation The first paragraph mentions the name of the formalism and its authors, 
as well as lists main papers. The year when the approach was proposed is given. Here we also 
present the main purpose for which the technique was introduced. If nothing is indicated about 
the formalism structure, it means that it is a generic DAG. If the structure is more specifically a 
tree, then it is indicated either in the formalism's name or in the first paragraph of the description. 

Main features In the second paragraph, we briefly explain the main features of the formalism, 
in particular what its added features are with respect to the state of the art at the time of 
its invention. Moreover, we state whether the modeling technique is formalized, i.e., whether it 
complies with proper mathematical definitions. 

Quantification Next, we focus on quantitative aspects of the considered methodology. We ex- 
plain whether the formalism is tailored for a couple of specific parameters or metrics, or whether a 
general framework has been introduced to deal with computations. In the first case, we list relevant 
attributes, in the second case, we briefly explain the new algorithms or calculation procedures. 

Practical aspects When relevant, we mention industrialized or prototype software tools sup- 
porting the described approach. We also indicate when real or realistic scenarios have been modeled 
and analyzed with the help of the described approach. In this paragraph, we also refer to large 
research projects and Ph.D. theses applying the methodology. This paragraph is optional. 

1 Different versions of the same paper (e.g., an official publication and a corresponding technical report) have 
been counted as the same publication. 
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Additional remarks We finish the formalism description by relating it to follow-up method- 
ologies. If it is the case, we point out the formalism's limitations that have been identified by its 
authors or other researchers from the field. In this part we also point out various other peculiarities 
related to the formalism. This paragraph is optional. 

3 Description of the Formalisms 

This section constitutes the main part of this survey. It describes numerous DAG-based approaches 
for graphical attack and defense modeling according to the template outlined in Section 2.3. 
Models gathered within each subsection are ordered chronologically, with respect to the year of 
their introduction. 

3.1 Static Modeling of Attacks 
3.1.1 Attack Trees 

Inspired by research in the reliability area, Weiss [286] in 1991 and Amoroso [9] in 1994 proposed 
to adopt a tree-based concept of visual system reliability engineering to security. Today, threat 
trees [9,103,157,255,265], threat logic trees [286], cyber threat trees [193], fault trees for attack 
modeling [253], and the attack specification language [261] can be subsumed under attack trees, 
which are AND-OR tree structures used in graphical security modeling. The name attack trees 
was first mentioned by Salter et al. in 1998 [230] but is often only attributed to Sclmeier and cited 
as [234,235]. 

In the attack tree formalism, an attacker's main goal (or a main security threat) is specified 
and depicted as the root of a tree. The goal is then disjunctively or conjunctively refined into sub- 
goals. The refinement is repeated recursively, until the reached sub-goals represent basic actions. 
Basic actions correspond to atomic components, which can easily be understood and quantified. 
Disjunctive refinements represent different alternative ways of how a goal can be achieved, whereas 
conjunctive refinements depict different steps an attacker needs to take in order to achieve a 
goal [218]. In 2005, Mauw and Oostdijk formalize attack trees by defining their semantics and 
specifying tree transformations consistent with their framework [161]. Kienzle and Wulf present 
an extensive general procedure for tree construction [127] while other researchers are engaged in 
describing how to generate attack tree templates using attack patterns [149, 178]. 

Quantification of security with the help of attack trees is a very active topic of research [288] . 
A first simple procedure for quantification of attack trees was proposed by Weiss [286] and is 
based on a bottom-up algorithm. In this algorithm, values are provided for all leaf nodes and the 
tree is traversed from the leaves towards the root in order to compute values of the refined nodes. 
Depending on the type of refinement, different functional operators are used to combine the values 
of the children. This procedure allows to analyze simple aspects, such as the costs of an attack, the 
time of an attack or the necessary skill level [2,9,18,21,43,74,92,97,98,147,161,229,230,234,257, 
286,288,296]. Whenever more complicated attributes, such as probability of occurrence, probability 
of success, risk or similarity measures are analyzed, additional assumptions, for example mutual 
independence of all leaf nodes, are necessary, or methods different from the bottom-up procedure 
have to be used [2, 31, 34, 37, 38, 43, 74, 97, 118, 147, 156, 193, 221, 226, 234, 278, 296, 298] . Propagation 
of fuzzy numbers that model fuzzy preference relations has initially been proposed in [25] and 
extended in [36]. Using Choquet integrals it is possible to take interactions between nodes into 
account. 

Commercial software for attack tree modeling, such as SecurlTrce [7] from Amenaza or At- 
tackTree+ [108] from Isograph provides a large database of attack tree templates. Academic tools, 
including SeaMonster [171] developed within the SHIELDS project [241] offer visualization and 
library support. Attack trees may occur in the SQUARE methodology [168]. The entire method- 
ology and therefore visualization of attack trees are supported by the SQUARE tool [248]. At- 
tackDog [141] was developed as a prototype software tool for managing and evaluating attack 
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trees with voting systems in mind but is believed to be much more widely applicable to eval- 
uating security risks in systems [3]. Numerous case studies [4,30,43,44,48,50,77,80,84,92,93, 
97, 98, 125, 142, 148, 158, 164, 165, 168, 178-180, 187, 199, 220, 229, 232, 254, 258, 259, 261, 284, 298] 
account for the applicability of the attack tree methodology. Attack trees are used in large inter- 
national research projects [81,241,263,266]. They have been focus of various Ph.D. and Master 
theses [35, 73, 79, 87, 94, 99, 116, 122, 126, 129, 152, 177, 185, 195, 196, 206, 214, 217, 223, 231, 233, 299]. 

Since attack trees only focus on static modeling and only take an attacker's behavior into 
account, numerous extensions that include dynamic modeling and a defender's behavior, exist. 
Except for formalisms involving Bayesian inference techniques, all other DAG-based formalisms 
refer back to the attack tree methodology. They point out a need for modeling defenses, dynamics, 
and ordered actions, as well as propose computation procedures for probability or highly specified 
key figures. Neither the name attack trees, nor the initial formalization of Mauw and Oostdijk is 
universally accepted. Some researchers consider attack trees, threat trees or fault trees to essentially 
be the same [11,104,181,251,253,267] while other researchers point out specific differences [147, 
174]. As common ground all mentioned methodologies use an AND-OR tree structure but are 
divided on what the tree can actually model (attacks, vulnerabilities, threats, failures, etc.) 

3.1.2 Augmented Vulnerability Trees 

Vulnerability trees [271] have been proposed by Vidalis and Jones in 2003 to support the decision 
making process in threat assessment. Vulnerability trees are meant to represent hierarchical inter- 
dependence between different vulnerabilities of a system. In 2008, Patel, Graham and Ralston [200] 
extended this model to augmented vulnerability trees which combine the concepts of vulnerability 
trees, fault tree analysis, attack trees, and cause-consequence diagrams. The aim of augmented 
vulnerability trees is to express the financial risk that computer-based information systems face, 
in terms of a numeric value, called "degree of security". 

The root of a vulnerability tree is an event that represents a vulnerability; the branches cor- 
respond to different ways of exploiting it. The leaves of the tree symbolize steps that an attacker 
may perform in order to get to the parent event. The model, which is not formally defined, uses 
only AND and OR connectors depicted as logical gates. Vulnerability trees are very similar to 
attack trees, they differ in how the root event is defined (vulnerability event vs. an attacker's 
goal) . A step- wise methodology consisting of a sequence of six steps is proposed in [200] to create 
an augmented vulnerability tree and analyze security related indexes. 

The authors of [271] propose a number of attributes on vulnerability trees, including: com- 
plexity value (the smaller number of steps that an attacker has to employ in order to achieve his 
goal), educational complexity (qualifications that an attacker has to acquire in order to exploit a 
given vulnerability), and time necessary to exploit a vulnerability. However, the paper [271] does 
not detail how to compute these attributes. In [200], the model is augmented with two indexes: 
the threat-impact index and the cyber-vulnerability index. The first index, represented by a value 
from [0, 100], expresses the financial impact of a probable cyber threat. The lower the index the 
smaller the impact from a successful cyber attack. The second index, also expressed by a value 
from [0, 100], represents system flaws or undesirable events that would help an intruder to launch 
attacks. The lower this index, the more secure the system is. 

In [257], the augmented vulnerability tree approach has been used to evaluate risks posed to a 
SCADA system exposed to the mobile and the Internet environment. 

3.1.3 Augmented Attack Trees 

In 2005, Ray and Poolsappasit 2 first published about augmented attack trees to provide a prob- 
abilistic measure of how far an attacker has progressed towards compromising the system [219]. 
This tree-based approach was taken up by H. Wang et al. in 2006 and extended to allow more 
flexibility in the probabilistic values provided for the leaf nodes [274]. When again publishing 
in 2007, Poolsappasit and Ray used a different definition of augmented attack trees to be able 

2 In early papers spelled Poolsapassit [212,219] 
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to perform a forensic analysis of log files [212]. Using the second definition of augmented attack 
trees, J. Wang et al. performed an analysis of SQL injection attacks [276] and DDoS attacks [275]. 
They also extended augmented attack trees further to measure the quality of detectability of an 
attack [277]. Co-authors of Dewri, namely Poolsappasit and Ray, formalized attack trees as AND- 
OR structure where every node is interpreted to answer a specific binary question [65,66]. This 
formalization is then again extended to augmented attack trees by adding to every node an indi- 
cator variable and an additional value with the help of which the residual damage is computed. 
On the enhanced structure they are able to optimize how to efficiently trade-off between spent 
money and residual damage. 

The various ways of defining augmented attack trees are based on attack trees (Section 3.1.1). 
In the first definition, attack trees are augmented by node labels that quantify the number of 
compromised subgoals on the most advanced attack path as well as the least-effort needed to 
compromise the subgoal on the most advanced path to be able to compute the probability of 
attack [219]. H. Wang et al. generalized this definition from integer values to general weights. 
Both approaches include tree pruning and tree trimming algorithms to eliminate irrelevant nodes 
with respect to intended operations (behavior) of a user [274]. In the second definition, attack 
trees are augmented by descriptive edge labels and attack signatures. Each edge defines an atomic 
attack which is described by the label and represents a state transition from a child node to 
the corresponding parent. An attack signature is a sequence of groups of incidents, from which a 
sequence of incidents can be formed, which executed constitutes an atomic attack. The sequences 
are then exploited to filter log files for relevant intrusion incidences [212] and used to describe state 
transitions in SQL injection attacks using regular expressions [276]. Moreover they are exploited 
to model state transition in DDoS attacks [275] and adapted to provide a measure for quality of 
service detection, called quality of detectability [277]. In an extension of the third definition [66] 
the system administrator's dilemma is thoroughly examined. The purpose of this extension is to 
be able to compute a bounded minimization of the cost of the security measures while also keeping 
the residual damage at a minimum. 

Augmented attack trees were designed with a specific quantitative purpose in mind. The first 
formalization of augmented attack trees was introduced to compute the probability of a system 
being successfully attacked. Additionally to increasing the descriptive capabilities of the method- 
ology, the second definition is accompanied by several algorithms that help compute the quality of 
detectability in [277]. As mentioned before, the third definition targets solving the system adminis- 
trator's dilemma. This is achieved by using a simplistic cost model a multi-objective optimization 
algorithm which guides the optimization process of which security hardening measures best to 
employ. 

The authors of the first formalism state that attempts by system administrators to protect the 
system will not change the outcome of their analysis. A similar shortcoming is suggested for the 
second formalization. 

3.1.4 OWA Trees 

In 2005, Yager proposed to extend the AND and OR nodes used in attack trees by replacing them 
with ordered weighted averaging (OWA) nodes. The resulting formalism is called OWA trees [296] 
and it forms a general methodology for qualitative and quantitative modeling of attacks. 

Regular attack trees make use of two (extreme) operators only: AND (to be used when all 
actions need to be fulfilled in order to achieve a given goal) and OR (to be used when the fulfillment 
of at least one action is sufficient to reach a desired result). OWA operators represent quantifiers 
such as most, some, half of, etc. Thus, OWA trees are well suited to model uncertainty and to 
reason about situations where the number of actions that need to be satisfied is unknown. OWA 
trees are static in the sense that they do not take interdependencies between nodes into account. 
They have been formally defined in [296] using the notion of an OWA weighting vector. Since AND 
and OR nodes can be seen as special cases of OWA nodes, mathematically, attack trees form a 
subclass of OWA trees. Therefore, algorithms proposed for OWA trees are also suitable for the 
analysis of attack trees. 
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In [296], Yager provides sound techniques for the evaluation of success probability and cost 
attributes on OWA trees. For the probability attribute, he identifies two approaches that can be 
explained using two different types of attackers. The first approach assumes that the attacker is 
able to try all available actions until he finds one that succeeds. Since in most situations such an 
assumption is unrealistic, the author proposes a second model, where an attacker simply chooses 
the action with the highest probability of success. Furthermore, [296] presents two algorithms 
for computing the success probability attribute: one assumes independent actions which leads to 
a simpler calculation procedure, the other can deal with dependent actions. Finally, the author 
discusses how to join the two attributes together, in order to correctly compute the cheapest and 
most probable attack. 

In [25], Bortot, Fedrizzi and Giove proposed the use of Choquet integrals in order to reason 
about OWA trees involving dependent actions. 

3.1.5 Parallel Model for Multi-Parameter Attack Trees 

In 2006, Buldas, Laud, Priisalu, Saarepera and Willemson initiated a series of papers on rational 
choice of economically relevant security measures using attack trees. The proposed model is called 
multi-parameter attack trees and was first introduced in [31]. Between 2006 and 2010, researchers 
from different research institutes in Estonia proposed six follow-up papers [32,117-119,186,289], 
extending and improving the original model proposed in [31]. 

Most of the approaches for quantitative analysis using attack trees, prior to [31], focus on one 
specific attribute, e.g., cost or feasibility of an attack. In reality, interactions between different 
parameters play an important role. The aim of the mentioned series of papers was to study how 
tree computations must be done when several interdependent parameters are considered. The 
model of multi-parameter attack trees assumes that the attacker behavior is rational. This means 
that attacks are considered unlikely if their costs are greater than the related benefits and that 
the attacker always chooses the most profitable way of attacking. The parallel model for multi- 
parameter attack trees has been studied in [31,32,116-119]. This model assumes that all elementary 
attacks take place simultaneously, thus the attacker does not base his decisions on success or failure 
of some of the elementary attacks. 

Multi-parameter attack trees concentrate on the attribute called expected attacker's outcome. 
This outcome represents a monetary gain of the attacker and depends on the following parame- 
ters: gains of the attacker in case the attack succeeds, costs of the attack, success probability of 
the attack, probability of getting caught and expected penalties in case of being caught. First, a 
game theoretical model for estimation of the expected attacker's outcome was proposed by Bul- 
das et al. [31], where values of all parameters are considered to be precise point estimates. In [117], 
Jiirgenson and Willemson extend the computation methods proposed in [31] to the case of interval 
estimations. Later it turned out that the computational model from [31] was imprecise and incon- 
sistent with the mathematical foundations of attack trees introduced in [161]. Hence, an improved 
approach for the parallel attack tree model was proposed by Jiirgenson and Willemson [118]. 
Since this new approach requires exponential running time to determine possible expected out- 
come of the attacker, an optimization solution, based on a genetic algorithm for fast approximate 
computations, has been proposed by the same authors in [119]. 

In [32], Buldas and Magi applied the approach developed in [31] to evaluate the security of 
two real e-voting schemes: the Estonian E-voting System in use at the time (EstEVS) and the 
Secure Electronic Registration and Voting Experiment (SERVE) performed in the USA in 2004. 
A detailed description of this case study is given in the Master thesis of Magi [152]. A prototype 
computer tool supporting the security analysis using the multi-parameter attack trees has been 
implemented [12] and described in [13]. 

In Section 3.2.9, we describe the serial model for multi-parameter attack trees, which extends 
the parallel model with an order on the set of elementary components. 
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3.1.6 Extended Fault Trees 

Extended fault trees were presented by Fovino et al. at the ESREL conference in 2007 [159] and 
published in an extended version as a journal paper [88] issued in 2009. The formalism aims at 
combining malicious deliberate acts, which are generally captured by attack trees (Section 3.1.1), 
and random failures, which are often associated with classical fault trees (Section 3.1.1). 

Extended fault trees and attack trees are structurally similar. The main difference between 
the two formalisms is in the type of basic events that can be modeled. In EFT basic events can 
represent both non-malicious, accidental failures as well as attack steps or security events. Basic 
events of attack trees usually correspond to malicious attacker's actions only. Logical AND and OR 
gates are explicitly represented in the same way as in classical fault trees. A step-by-step model 
construction process is described in [88], defining how existing fault-trees can be extended with 
attack-related components to form extended fault tree models. The modeling technique complies 
with proper mathematical foundations, directly issued from fault trees as defined in the safety and 
reliability area. 

Quantification capabilities are focused on the computation of the probability of occurrence 
of the top-event (root node). Generic formulas from fault tree quantitative analysis are recalled 
in [88], including treatment of independent or mutually exclusive events. However, no concrete 
examples of quantification are provided. 

A simple example, analyzing the different failure and attack scenarios leading to the release of 
a toxic substance by a chemical plant, is described in [88]. No particular tool has been developed 
to support extended fault trees, however, all classical fault tree tools may be used directly. 

One of the limitations explicitly stressed by the inventors of extended fault trees is that they 
do not take into account time dynamics. 

3.2 Sequential Modeling of Attacks 

3.2.1 Cryptographic DAGs 

Meadows described cryptographic DAGs in 1996 (proceedings published in 1998), in order to 
provide a simple representation of an attack process [169]. The purpose of the formalism is limited 
to visual description. The attack stages of the overall attack process correspond to the nodes of a 
DAG. The difficulty of each stage is shown by a color code. In 1996, the novelty of cryptographic 
DAGs was to provide a simple representation technique of sequences and dependencies of attack 
steps towards a given attacker's objective. 

From a modeling point of view, each stage (represented as a colored box) contains a textual 
description of atomic actions needed for the realization of the stage. Arrows represent depen- 
dencies between the boxes. A simple arrow indicates that one stage is needed to realize another 
stage. Two arrows fanned out symbolize that one stage enables another one repeatedly. More 
generally speaking, cryptographic DAGs are an informal formalism targeted at high level system 
descriptions. 

Cryptographic DAGs do not support any type of quantification. 

Cryptographic DAGs have been used in [169] to demonstrate attacks on cryptographic protocols 
(with SSL and Needham-Schroeder scheme as a use-cases) , however this representation technique 
may be used to model other types of attacks as well. 

This formalism allows the representation of sequences of attack steps, and dependencies be- 
tween those steps, but cannot capture "static" relations like "AND" and "OR". Moreover, the clarity 
and usability of the models depends heavily on the text inside the boxes, which is not standardized. 

3.2.2 Fault Trees for Security 

Fault tree analysis was born in 1961 and has initially been developed into a safety, reliability 
and risk assessment methodology [107,252,270,285]. A short history of non-security related fault 
trees was published by Ericson II [78] in 1999. Fault trees have also been adopted to software 
analysis [95,96, 145, 146] and were even equated with attack trees by Steffen and Schumacher [253]. 
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In 2003, however, Brooke and Paige adopted fault tree for security, extending the classical AND-OR 
structure of attack trees (Section 3.1.1), to include well known concepts from safety analysis [29]. 

Based on an AND-OR structure, three additional connectors (priority AND, exclusive OR and 
inhibit), specific node types (basic, conditioning, undeveloped, external and intermediate) as well 
as transfer symbols (transfer in, transfer out) to break up larger trees are adopted from fault tree 
analysis in its widest sense. Fault trees for security are an aid to the analysis of security-critical 
systems, where first an undesired (root) event is identified. Then, new events are constructed by 
inserting connectors that explicitly identify the relationship of the events to each other. Several 
rules, like the "no miracle" rule, the "complete the gate" rule and the "no gate to gate" rule 
are adopted directly from fault trees. Construction stops when there are no more uncompleted 
intermediate events. In the end, a completed fault tree serves as an "attack handbook" by providing 
information about the interactions by which a security critical system fails. 

In [29] , Brooke and Paige state that in computer security "it is difficult to assign useful prob- 
abilities to the events". Consequently probabilistic quantitative analysis is debatable. Instead the 
authors recommend to perform risk analysis which answers how the system fails based on the 
primary events (leaf nodes) . 

While [29] only provides a toy example, the authors state that any tool used in fault tree 
analysis can be used. They refer to [64] as a good overview of available programs. 

3.2.3 Bayesian Networks for Security 

Starting in 2004, different researchers proposed, seemingly independently, to adopt Bayesian net- 
works, whose origin lies in artificial intelligence, as a security modeling technique [111,184,201,202]. 
Bayesian networks are also known as belief network or causal network. In Bayesian networks, nodes 
represent events or objects and are associated with probabilistic variables. Directed edges represent 
causal dependencies between nodes. Mathematical algorithms developed for Bayesian networks are 
suited to solve probabilistic questions on DAG structures. They are aimed at keeping the exponent 
small when the computing algorithm is exponential and reduce to polynomial algorithms if the 
DAG is actually a tree. 

According to Qin and Lee, the objective of Bayesian Networks for Security is to "use prob- 
abilistic inference techniques to evaluate the likelihood of attack goals (s) and predict potential 
upcoming attacks" [218]. They proposed the following procedure that converts an attack tree into 
a Bayesian network. Every node in the attack tree is also present in the Bayesian network. An "OR" 
relationship from an attack tree is modeled in the Bayesian network with edges pointing from refin- 
ing nodes that represent causes into the corresponding refined nodes that represent consequences. 
Deviating from regular attack trees, an "AND" relationship is assumed to have an explicit (or 
implicit) order in which the actions have to be executed. This allows to model the "AND" relation- 
ship by a directed path, which starts from the first (according to the order) child and ends with 
the parent node. Dantu et al. follow a different strategy when using Bayesian networks to model 
security risk management starting from behavior-based attack graphs 3 [58-61]. When process- 
ing multi-parameter attack trees with estimated parameter values (Section 3.1.5) Jiirgenson and 
Willemson use Qin and Lee's conversion of an attack tree to a Bayesian network [117]. An et al. 
propose to add a temporal dimension and to use dynamic Bayesian networks for intrusion detec- 
tion without specifying how the graph is set up [10]. Althebyan and Panda use knowledge graphs 
and dependency graphs as basis for the construction of a Bayesian network [6]. They analyze a 
specific type of insider attack and state that their computational procedures were inspired by 
Dantu et al. Another approach involving Bayesian networks is described by Xie et al. who ana- 
lyze intrusion detection systems [294]. They state that the key to using Bayesian networks is to 
"correctly identify and represent relevant uncertainties" which governs their setup of the Bayesian 
network. 

3 The authors do not appear to make a distinction between attack trees and attack graphs. Since their method- 
ology is only applicable to cycle-free structures and they do not mention how to deal with cycles, we assume that 
the methodology is actually based on attack DAGs or attack trees. 
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Bayesian networks are used to analyze security under uncertainty. The DAG structure is of 
great value because it allows to use efficient algorithms. On the one hand there exist efficient 
inference algorithms that compute a single query (variable elimination, bucket elimination and 
importance, which are actually equivalent according to Pouly and Kohlas [215]) and on the other 
hand there are inference algorithms that compute multiple queries at once (bucket tree algorithm 
and Lauritzen-Spiegelhalter algorithm). In fact, the efficiency of these algorithms can be seen as 
main reason to the success of Bayesian networks, since querying general graphs is an NP-hard 
problem [16,24]. Another strength of Bayesian networks is their ability to update the model, i.e., 
compute a posteriori distribution, when new information is available. 

We have not found any dedicated tools for analysis of Baysian networks for security. However, 
numerous tools exist that allow a visual treatment of standard Bayesian networks. One such tool 
is the Graphical Network Interface (GeNIE) that uses the Structural Modeling, Inference, and 
Learning Engine (SMILE) [63]. It was, for example, used in [183] to analyze the interoperability 
of a very small cluster of services and mentioned as hypothetical use in [89] . Another one, called 
MulVal [198], was actually developed for attack graphs (Section 5.2), but used in [294] to implement 
a Bayesian network model. A third tool, tailored to statistical learning with Bayesian networks is 
bnlearn [236]. 

There also exist isolated papers that promote the use of Bayesian networks in security without 
any relation to attack trees or attack graphs. Houmb et al., quantify security risk level from 
Common Vulnerability Scoring System (CVSS) estimates of frequency and impact using Bayesian 
networks [102]. Feng and Xie also use Bayesian networks and provide an algorithm of how to 
merge two sources of information, expert knowledge and information stored in databases, into one 
graph [83] . Note that in this section we have gathered approaches that rely on Bayesian networks 
and where their construction starts from graphs that do not contain any cycles. Graphical models 
that make use of Bayesian networks and initially contain cycles are treated in Section 3.2.4, ones 
that include defenses are treated in Section 3.4.3. 

3.2.4 Bayesian Attack Graphs 

Bayesian Attack Graphs combine (general) attack graphs (Section 5.2), with computational pro- 
cedures of Bayesian networks (Section 3.2.3). However, since Bayesian inference procedures only 
work on cycle-free structures, the formalism includes instructions on how to remove any occurring 
cycles. Hence any final Bayesian attack graph is acyclic. After the elimination of cycles, Bayesian 
attack graphs model causal relationships between vulnerabilities in the same way as Bayesian 
networks (Section 3.2.3) Bayesian attack graphs were first proposed by Liu and Man in order to 
analyze network vulnerability scenarios with the help of Bayesian inference methods in 2005 [151]. 
Therefore the formalism advances computational methods in security where uncertainty is consid- 
ered. 

The formalism of Man and Liu is not the only fusion of attack graphs and Bayesian networks. 
Starting in 2008 a group of researchers including Frigault, Noel, Jajodia and Wang published a 
paper on a modified version of Bayesian attack graphs. Their goal was to be able to calculate 
general security metrics regarding information system networks which also contain probabilistic 
dependencies [90,192]. Later they extended the formalism, using a second copy of the model as 
time slice, to also capture dynamic behavior in so called dynamic Bayesian networks [91]. In 2012, 
Poolsappasit et al. revisited the framework to be able to deal with asset identification, system 
vulnerability and connectivity analysis as well as mitigation strategies [213]. 

All three approaches eliminate cycles that possibly exist in the underlying attack graph. A 
shortcoming of Liu and Man is that they do not provide a specific procedure on how to achieve 
this. The group including Frigault refers to a paper on attack graphs [279] which removes cycles 
through an intricate procedure. Poolsappasit et al. state that they rather analyze "why an attack 
can happen" and not "how an attack can happen" and therefore, "cycles can be disregarded using 
the monotonicity constraint" mentioned in [8[. 

Since Bayesian attack graphs are cycle- free, evaluation on them can make use of Bayesian infer- 
ence techniques. For this it is necessary to provide probabilistic information. The three approaches 
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differ in how they compute quantitative values. Liu and Man provide edge probabilities [151], 
Frigault et al. give conditional probability tables for nodes which are estimated according to 
their CVSS score [90] and Poolsappasit et al. use (local) conditional probability distributions for 
nodes [213]. Furthermore, Poolsappasit et al. augment Bayesian attack graphs with additional 
nodes and values representing hardening measures (defenses). On the augmented structure they 
propose a genetic algorithm that solves a multiobjective optimization problem of how to assess 
the risk in a network system and select optimal defenses [213]. 

The research group including Wang uses a Topological Vulnerability Analysis (TVA) tool [110, 
188] to create the attack graphs that serve as basis for constructing Bayesian attack graphs. 
Poolsappasit et al. have developed an unreferenced in-house tool that allows them to compute 
with conditional probability distributions. 

Wang et al. [90,91] state that their work is also based on that of An et al. [10], who use Bayesian 
networks without cycles for modeling risks of violating privacy in a database. 

3.2.5 Compromise Graphs 

McQueen et al. introduced compromise graphs in 2006 [167]. Compromise graphs are based on 
directed graphs 4 , and are used to assess the efficiency of various technical security measures for a 
given network architecture. The nodes of a compromise graph represent the phases of an attack, 
detailing how a given target can get compromised. The edges are weighted according to the esti- 
mated time required to complete the corresponding phase for this compromise. The overall time 
needed for the attacker to succeed is computed and compared along different defensive settings, 
providing a metric to assess and compare the efficiency of these different defensive settings. 

The formalism has a sound mathematical formalization: a time to compromise (TTC) metric is 
modeled for each edge as a random process combining three sub-processes. Each of these processes 
has a different probability distribution (mixing exponential, gamma and beta-like distributions). 
The value for the process model parameters are based on the known vulnerabilities of the consid- 
ered component and the estimated skill of the attacker. A complete description and justification 
of such a stochastic modeling is provided by the same authors in a previous paper [166]. In com- 
promise graphs, five types of stages, corresponding to the vertices of the graph, are modeled: 
recognition, breaching the perimeter, penetration, escalation of privilege, damage. 

Compromise graphs are used to evaluate the efficiency of security measures, such as system 
hardening, firewalls or enhanced authentication. This is achieved by comparing the shortest paths 
(in terms of TTC) of compromise graphs with and without such measures in place. 

The approach is illustrated in [167] by modeling attacks on a SCADA system. 

Byres and Leversage adopt a very similar approach in [143,144], called state-time estimation 
algorithm (STEA), directly inspired by McQueen et al. They combine a slightly modified TTC 
calculation approach with a decomposition of the attack according to the architectural areas of 
the targeted system. 

3.2.6 Enhanced Attack Trees 

Enhanced attack trees have been introduced by C am tepe and Yener to to support an intrusion 
detection engine by modeling complex attacks with time dependencies. This model was first de- 
scribed in a technical report [45] in 2006. One year later, an official conference proceedings [46] 
appeared. 

In addition to classical OR and AND gates, enhanced attack trees rely on the use of a new 
gate, the "ordered- AND", which allows to capture sequential behavior and constraints on the 
order of attack steps. The model of enhanced attack trees has sound mathematical foundations. 
Additionally to the formalism description, [46] devises a new technique for detection of attacks. 
The new technique is based on automata theory and it allows to verify completeness of enhanced 
attack tree models with respect to the observed attacks. 

4 The authors do not state whether these directed graphs are acyclic or not, but the description of compromise 
graphs and their examples led us to consider compromise graphs as DAGs. 
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The quantification capabilities described in [46] are directly related to intrusion detection 
(probability of a given attack occurring based on a set of observed events). A confidence attribute 
measured in percent is denned for subgoals as "the chance of reaching the hnal goal of the attacker 
when a subgoal is accomplished". It is computed as the ratio of all accomplished events until a 
subgoal is realized, over all events of the modeled scenario. This attribute aims at supporting 
an early warning system, supporting decision-making and reaction before actual damages occur. 
Moreover, [46] introduces an original parameter called "time to live" which allows to express that 
some steps are only available in a given time window. 

In [175], Mishra et al. also make use of ordered- AND operators, referring to [46]. The au- 
thors visually describe Stuxnet and similar attacks, but do not use Qamtepe and Yener's rigorous 
formalization to analyze the models. 

3.2.7 Vulnerability Cause Graphs 

Vulnerability cause graphs were invented in 2006 by Ardi, Byers and Shahmehri as a key element 
of a methodology that supports security activities throughout the entire software development 
lifecycle [15]. 

The formalism can be seen as a root cause analysis for security-related software failures, be- 
cause it relates vulnerabilities with their causes. In a vulnerability cause graph, every node except 
for one, has an outgoing directed edge. The single node without a successor is called the exit node 
and represents the considered vulnerability. All other nodes represent causes. The predecessor- 
successor (parent-child) relationship shows how certain conditions (nodes) might cause other con- 
ditions (nodes) to be a concern. In an improved version of vulnerability cause graphs [39], nodes 
can be simple, compound or conjunctions. Simple nodes represent conditions that may lead to a 
vulnerability. Compound nodes facilitate reuse, maintenance and readability of the models. Con- 
junctions represent groups of two or more other nodes. Contrary, disjunctions occur if a node has 
two or more predecessors. In this case, the original nodes might have to be considered if either of 
its predecessors might have to be considered. Finally, if the causes have to follow a certain order, 
they are modeled as sequences of nodes. To construct a vulnerability cause graph, the exit node 
is considered as a starting point and refined with causes. 

In vulnerability cause graphs, nodes can be annotated as "blocked" if the underlying causes are 
mitigated. The "blocked" flag allows the user to compute whether the underlying vulnerability (exit 
node) is also mitigated. Vulnerability cause graphs are also equipped with a notion of graph trans- 
formations that do not change whether the vulnerability is mitigated or not. The transformations 
include conversions of conjunctions, reordering of sequences, combination of nodes, conversion to 
compound nodes as well as derived transformations. 

In [39] the vulnerability CVE-2003-0161, in [40] the vulnerability CVE-2005-2558, and in [154] 
the vulnerability CVE-2005-3192 is analyzed with the help of VCGs. Furthermore, [47] contains 
an additional three case studies on common software vulnerabilities which have been performed 
using VCGs. The SHIELDS project [241] has developed a software tool GOAT [240] to be used in 
conjunction with vulnerability cause graphs. 

Vulnerability cause graphs were developed as part of a comprehensive methodology to reduce 
software vulnerabilities that arise in ad hoc software development. They are the starting point 
to build security activity graphs (Section 3.3.4). By introducing compound nodes, the inventors 
of the formalism have created a model that allows different layers of abstraction, which in turn 
introduced a problematic design decision of how many layers of abstraction are needed. 

3.2.8 Dynamic Fault Trees 

In 2009, Khand [124] adapted several dynamic fault tree [69, 70] gates to attack trees, in order to 
add a dynamic dimension to classical attack trees. The aim of the formalism are similar to those 
of attack trees (Section 3.1.1). 

Dynamic fault trees [69, 70] were invented by Dugan et al. in the early 1990s to overcome 
limitations of static fault trees. They aim at combining the dynamic capacities of Markovian 
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models with the "look and feel" of fault trees. To achieve this, four dynamic gates are used: the 
"priority-AND" (PAND), the "sequence gate" (SEQ), the "functional dependency gate" (FDEP) and 
the "cold spare gates" (CSP). Khand reuses directly the three first gates (although renaming FDEP 
gates by CSUB, for Conditional Subordination, gates), leaving out the CSP gates. The PAND gate 
reaches a success state if all of its input are realized in a pre-assigned order (from left to right in 
the graphical notation). The SEQ gate allows to model a dependency between events, such that 
these events can only be realized in a particular order (from left to right in the graphical notation) . 
Once all the input events are realized, the gate is verified. The CSUB gate models the need of 
the realization of a trigger event to allow a possible realization of others events. Dynamic fault 
tree combines dynamic gates with classical logical gates (AND, OR). Dynamic gates are formally 
defined with truth tables in [124], and by Markov processes in the general definitions of dynamic 
fault trees from the safety literature [69,70] (although the description is still incomplete [26]). 
There is no quantification aspects developed in [124]. 

The paper by Khand does not specify which tool to use in order to treat the models, but several 
tools exist for dynamic fault trees in the reliability area, e.g., Galileo [71]. 

In safety studies, quantifications associated with dynamic fault trees are usually made using 
Markovian analysis techniques; those might be used here also although nothing is said about 
computation aspects. 

3.2.9 Serial Model for Multi-Parameter Attack Trees 

In 2010, the parallel model for multi-parameter attack trees (Section 3.1.5) has been extended by 
adding a temporal order on the set of elementary attacks [289] . This new methodology is called 
serial model for multi-parameter attack trees and was studied further in [116,186] and [33]. 

The model described in [116] and [186] assumes that an adversary performs the attacks in a 
given prescribed order. In [33], the authors introduce so called fully-adaptive adversary model, 
where an attacker is allowed to try atomic attacks in an arbitrary order which is not fixed in 
advance and can be modified based on the results of the previous trials. In both cases, the serial 
approach allows for a more accurate modeling of an attacker's behavior than the parallel approach. 
In particular, the attacker can skip superfluous elementary attacks and base his decisions on success 
or failure of the previously executed elementary attacks. 

In [289], an efficient algorithm for computing an attacker's expected outcome assuming a given 
order of elementary attacks is provided. Taking temporal dependencies into account allows the 
attacker to achieve better expected outcome than when the parallel model (Section 3.1.5) is used. 
As remarked in [119], finding the best permutation of the elementary attacks in the serial model 
for multi-parameter attack trees may turn computing the optimal expected outcome into a super- 
exponential problem. In [186], Niitsoo proposed a decision-theoretical framework which makes 
possible to compute the maximal expected outcome of a goal oriented attacker in linear time. 
In [33], Buldas and Stepanenko propose a game theoretical framework to compute upper bounds 
of the utility of fully-adaptive adversaries. 

A prototype computer tool supporting the security analysis using the serial model of multi- 
parameter attack trees has been implemented [12] and described in [13]. 

A thorough comparison of the parallel and the serial model for multi-parameter attack trees 
has been given in the Ph.D. thesis of Jiirgenson [116]. Baca and Petersen mention that in order to 
use parametrized attack trees, the user needs to have a good understanding of the motivations of 
the attacker [18]. To overcome this difficulty cumulative voting is used in countermeasure graphs 
(Section 3.3.7). 

3.2.10 Improved Attack Trees 

Improved attack trees are aim at dealing with security risks that arise in space-based information 
systems. They were proposed by Wen-ping and Wei-min in 2011 [287] to more precisely describe 
attack on the information transmitting links, acquisitions systems and ground-based supporting 
and application systems. 
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The formalism is based on attack trees and explicitly incorporates the use of the sequential AND 
operator. It is not defined in a formal way. Improved attack trees rely heavily on the description 
by Schneier and only detail how to specifically compute the system risk. 

Improved attack trees provide a specific formula to evaluate a risk value for each leaf node. 
Starting from these risk values, the risk rate and the risk possibility are computed and multiplied 
to compute the overall system risk. The formulas distinguish between OR, AND and sequential 
AND nodes. 

3.3 Static Modeling of Attacks and Defenses 

3.3.1 Anti-Models 

Anti-models [268] have been introduced by Lamsweerde et al. in 2003. They are closely related to 
AND-OR goal-refinement structures [269] (sometimes called goal models) used for goal analysis in 
requirements engineering. Anti-models extend such AND-OR goal-refinement structures with the 
possibility to model malicious and intentional obstacles to security goals, called anti-goals. They 
can be used to generate subtle attacks, discard non-realizable or unlikely ones, and derive more 
effective customized resolutions. 

In [268] and later in an extended version [267], van Lamsweerde et al. provide a six steps pro- 
cedure for a systematic construction of anti-models. First, anti-goals, representing an attacker's 
goals, are obtained by negating confidentiality, privacy, integrity, availability, authentication or 
non-repudiation requirements. For each anti-goal, the questions "who" and "why" are asked to 
identify potential classes of attackers and their higher-level anti-goals. An AND-OR refinement 
process is then applied to reach terminal anti-goals that are realizable by the attackers. The result- 
ing AND-OR anti-models relate "attackers, their anti-goals, referenced objects and anti-operations 
(necessary to achieve their anti-goals) to the attackees, their goals, objects, operations and vulner- 
abilities." The construction of anti- models is only informally presented in [268] . Formal techniques 
developed for AND-OR goal-refinement structures (such as refinement obstacle trees) [269] can 
be used for the generation and analysis of anti-models. In particular, real-time temporal logic can 
be employed to model anti-goals as sets of attack scenarios. After identifying possible anti-goals, 
countermeasures expressed as epistemic extensions of real-time temporal logic operators are se- 
lected based on severity or likelihood of the corresponding threat and non-functional system goals 
that have been identified earlier. Possible resolutions tactics, inspired by solutions proposed for 
analysis of non- functional requirements in software engineering, are described in [269] and [267]. 
Applying resolution operators yields new security goals to be integrated in the model. These new 
goals are then again refined with the help of AND-OR structures. These, in turn, may require a 
new round of anti-model construction and analysis. 

The anti-models do not include quantitative analysis of security goals or anti-goals. 

3.3.2 Defense Trees 

Defense trees 5 are attack trees where leaf nodes are decorated with a set of countermeasures. 
They have been introduced by Bistarelli et al. in 2006 [22]. The approach combines qualitative 
and quantitative aspects and serves general security modeling purposes. 

The approach proposed by Bistarelli et al. was a first step towards integrating a defender's 
behavior into models based on attack trees. The analysis methodology for defense trees proposed 
in [22] and [21] uses rigorous and formal techniques, such as calculation of economic indexes and 
game theoretical solution concepts. However, the model itself is only introduced verbally and a 
formal definition is not given. 

In [22], the return on attack (ROA) and return on investment (ROI) indexes are used for quan- 
titative analysis of defense trees from the point of view of an attacker and a defender, respectively. 
The calculation of ROI and ROA is based on the following parameters: costs, impact, number of 
occurrences of a threat and gain. The indexes provide a useful method to evaluate IT security 

5 Papers by Bistarelli et al. use British English, thus originally, the name of their formalism is defence trees. 
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investments and to support the risk management process. In [21], game theoretical reasoning was 
introduced to analyze attack-defense scenarios modeled with the help of defense trees. In this 
paper, a defense tree represents a game between two players: an attacker and a defender. The ROI 
and ROA indexes, are used as utility functions and allow to evaluate the effectiveness and the 
profitability of countermeasures. The authors of [21] propose using Nash equilibria to select the 
best strategy for the players. 

In [23], defense trees have been extended to so called CP-defense trees, where modeling of 
preferences between countermeasures and actions is possible. Transforming CP-defense trees into 
answer set optimization (ASO) programs, allows to select the most suitable set of countermeasures, 
by computing the optimal answer set of the corresponding ASO program. Formalisms such as 
attack-defense trees (Section 3.3.6), and attack countermeasure trees (Section 3.3.5) extended 
defense trees by allowing defensive actions to be placed at any node of the tree and not only at 
the leaf nodes. 

3.3.3 Protection Trees 

Protection trees are a tree-based formalism which allow a user to allocate limited resources towards 
the appropriate defenses against specified attacks. The methodology was invented by Edge et al. 
in 2006, in order to incorporate defenses in the attack tree methodology [74]. 

Protection tree are similar to attack trees since both decompose high level goals into smaller 
manageable pieces by means of an AND-OR tree structure. The difference is that in protection 
tree the nodes represent protections. A protection tree is generated from an already established 
attack tree by finding a protection against every leaf node of the attack tree. Then the attack tree 
is traversed in a bottom-up way and new protection nodes are added to the protection tree if the 
protection nodes do not already cover the parent attack node. 

The AND-OR structure of protection trees is enriched with three metrics, namely probability 
of success, financial costs and performance costs on which the standard bottom-up approach is 
applied [73-75]. In [56], an additional metrics, the impact, helps to further prioritize where budget 
should be spent. 

The formalism has been investigated in case studies on how the U.S. Department of Homeland 
Security can allocate resources to protect their computer networks [74] , how an attack on an online 
banking system can be mitigated cost-efficiently [75], how to cheaply protect against an attack 
on computer and RFID networks [56] as well as a mobile ad hoc network [73]. When evaluating 
which defenses to install, the authors propose to first prune the tree according to the attacker's 
assumed capabilities. A larger, more applied case study to "evaluate the effectiveness of attack and 
protection trees in documenting the threats and vulnerabilities present in a generic Unmanned 
Aerial Systems (UAS) architecture" was performed by Cowan et al. [51]. 

In [75] a slightly different algorithm for creation of a protection tree was proposed. Here a 
designer starts by finding defenses against the root of an attack tree instead of the leaves, as 
in [73,74]. An approach similar to protection trees has been proposed in [228] to deal with the 
problem of threat modeling in software development. The paper uses so called identification trees 
to identify threats in software design and introduces the model of mitigation trees to describe 
countermeasures for identified threats. Despite an obvious modeling analogy between protection 
trees and mitigation trees, no connection between the two models has been made explicit in the 
literature. 

3.3.4 Security Activity Graphs 

In 2006, Ardi, Byers and Shahmehri introduced a formalism called security activity graphs (SAGs). 
The methodology was invented in order to "improve security throughout the software development 
process" [15]. SAGs depict possible vulnerability cause mitigations and are algorithmically gener- 
ated from vulnerability cause graphs (Section 3.2.7). 

SAGs are a graphical representation of first order predicate calculus and are based very loosely 
on ideas from fault tree analysis. In [15] the root of a SAG is associated with a vulnerability, 
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taken from a vulnerability cause graph. The vulnerability mitigations are modeled with the help 
of activities (leaf nodes). The syntax furthermore consists of AND-gates, OR-gates and split gates. 
The AND and OR-gates strictly follow Boolean logic, whereas the split gate allows one activity to 
be used in several parent activities, essentially creating a DAGs structure. The syntax of SAGs was 
changed in [41] for a more concise illustration of the models. Split gates no longer appear in the 
formalism. The functionality that simple activities can be distinguished from compound activities 
(complex activities that may require further breakdown) was added. Moreover cause references 
(possible attack points) serve as placeholders for a different SAG associated with a particular 
cause. 

In the SAG model, Boolean variables attached to the leaves of the SAG. A Boolean variable 
corresponding to an activity is true when it "is implemented perfectly during software development" 
otherwise, it is false. Then a value corresponding to the root of the SAG is deduced in a bottom-up 
fashion according to Boolean logic. 

Visual representation of SAGs is supported by SeaMonster [173] and GOAT [240]. Furthermore, 
SAGs have been used in [40, 41] to model the vulnerability CVE-2005-2558 in MySQL that leads 
to "denial of service or arbitrary code execution". 

Even though the model was devised in order to aid the software development cycle, the au- 
thors explicitly state that SAGs "lend themselves to other applications such as process analysis." 
SAGs are the middle step of a broader 3-steps approach for secure software development, with 
vulnerability cause graphs as a first step, and process component definition as a final step. In 2010 
SAGs were replaced by security goal models (Section 3.4.7) 

3.3.5 Attack Countermeasure Trees 

In 2010, Roy, Kim and Trivedi proposed attack countermeasure trees (ACT) [224,225] as a method- 
ology for attack and defense modeling which unifies analysis methods proposed for attack trees 
(Section 3.1.1) with those introduced on defense trees (Section 3.3.2). The main difference of ACTs 
with respect to defense trees is that in ACTs defensive measures can be placed at any node of the 
tree. Also, the quantitative analysis proposed for defense trees is extended by incorporating prob- 
abilistic analysis into the model. ACTs were first introduced in [225] and then further developed 
in [226]. 

ACTs may involve three distinct classes of events: attack events, detection events and mitigation 
events. The set of classical AND and OR nodes, as defined for attack trees, is extended with the 
possibility of using fc-out-of-n nodes. Generation and analysis of attack countermeasure scenarios is 
automated using minimal cut sets (mincuts) . Mincuts help to determine possible ways of attacking 
and defending a system and to identify the system's most critical components. 

A rigorous mathematical framework is provided for quantitative analysis of ACTs in [225] 
and [226]. The evaluation of the ROI and ROA attributes, as proposed for defense trees (Sec- 
tion 3.3.2), has been extended by adding the probability of attack, detection and mitigation events. 
The authors of [226] provide algorithms for probability computation on trees with an without re- 
peated nodes. With the help of probability parameters, further metrics, including cost, impact, 
Birnbaum's importance measure and risk, are evaluated. The use of the Birnbaum's importance 
measure (also called reliability importance measure, in the case of fault trees) is used to prioritize 
defense mechanisms countering attack events. Furthermore, in [226], Roy et al. propose a cubic 
algorithm to select an optimal set of countermeasures for an ACT. This addresses the problem 
of state-space explosion that the intrusion response and recovery engine based on attack-response 
trees (Section 3.4.5) suffers from. Finally in [227] the problem of selecting an optimal set of 
countermeasures with and without having probability assignments has been discussed. 

The authors of [226] implemented a module for automatic description and evaluation of ACTs 
in a modeling tool called Symbolic Hierarchical Automated Reliability and Performance Evalu- 
ator [264]. This implementation uses already existing algorithms for analysis of fault trees and 
extends them with algorithms to compute costs, impact and risk. Case studies concerning attacks 
on the Border Gateway Protocol (BGP), SCADA systems and malicious insider attacks have been 
performed using ACTs, as described in the Master thesis of Roy [223]. 
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The model of attack countermeasure trees is very similar to attack-defense trees. The main 
differences between the two models are listed in Section 3.3.6. 

3.3.6 Attack Defense Trees 

Attack-defense trees (ADTrees) were proposed by Kordy et al. in 2010 [132]. They allow to illus- 
trate security scenarios that involve two opposing players: an attacker and a defender. Consequently 
it is possible to model interleaving attacker and defender actions qualitatively and quantitatively. 
ADTrees can be seen as merging attack trees (Section 3.1.1) and protection trees (Section 3.3.3) 
into one formalism. 

In ADTrees, both types of nodes, attacks and defenses, can be conjunctively as well as disjunc- 
tively refined. Furthermore, the formalism allows for each node to have one child of the opposite 
type. Children of opposite type represent countermeasures. These countermeasures can be refined 
and countered again. Two sets of formal definitions build the basis of ADTrees: a graph-based 
definition and an equivalent term-based definition. The graph-based definition ensures a visual 
and intuitive handling of ADTrees models. The term-based representation allows for formal rea- 
soning about the models. The formalism is enriched through several semantics that allow to define 
equivalent ADTree representations of a scenario [133]. The necessity for multiple semantics is mo- 
tivated by diverse applications of ADTrees, in particular unification of other attack tree related 
approaches and suitability for various kinds of computations. In [135], the authors showed that, for 
a wide class of semantics (i.e., every semantics induced by a De Morgan lattice), ADTrees extend 
the modeling capabilities of attack trees without increasing the computational complexity of the 
model. In [133] the most often used semantics for ADTrees have been characterized by finite axiom 
schemes, which provides an operational method for defining equivalent ADTree representations. 
The authors of [131], have established a connection between game theory and graphical security 
assessment using ADTrees. More precisely, ADTrees under a semantics derived from prepositional 
logics are shown to be equally expressive as two-player binary zero-sum extensive form games. 

The standard bottom- up algorithm, formalized for attack trees in [161], has been extended 
to ADTrees in [133]. This required the introduction of four new operators (two for conjunction 
and disjunction of defense nodes and two for countermeasure links) [133]. Together with the two 
standard operators (for conjunctions and disjunctions of attack nodes) and a set of values, the 
six operators form an attribute domain. Specifying attribute domains allows the user to quantify 
a variety of security relevant parameters, such as time of attack, probability of defense, scenario 
satisfiability and environmental costs. The authors of [133] show that every attribute for which 
the attribute domain is based on a semi-ring can be evaluated on ADTrees using the bottom-up 
algorithm. How to properly specify attribute domains in terms of questions in natural language 
was presented in [134]. 

An extensive case study on a real-life RFID goods management system was performed by 
academic and industrial researchers with different backgrounds [19]. The case study resulted in 
specific guidelines about the use of attributes on ADTrees. A software tool, called the ADTool [136], 
supporting the attack-defense tree methodology, has been developed as one of the outcomes of 
the ATREES project [17]. The main features of the tool are easy creation, efficient editing, and 
quantitative analysis of ADTrees [130]. Since from a formal perspective, attack trees (Section 3.1.1), 
protection trees (Section 3.3.3), and defense trees (Section 3.3.2) are instances of attack-defense 
trees, the ADTool also supports all these formalisms. 

Finally, ADTrees can be seen as natural extension of defense trees (Section 3.3.2) where defenses 
are only allowed as leaf nodes. The ADTree formalism is quite similar to attack countermeasure 
trees (Section 3.3.5), however, there exist a couple of fundamental differences between the two 
models. On the one hand, in ADTrees defense nodes can be refined and countered, which is 
not possible in attack countermeasure trees. On the other hand, attack countermeasure trees 
distinguish between detection and mitigation events which are both modeled with defense nodes 
in ADTrees. Another difference is that attack countermeasure trees are well suited to compute 
specific parameters, including probability, return on investment (ROI) and return on attack (ROA). 
ADTrees, in turn, focus on general methods for attribute computation. A different formalism, also 
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called attack-defense trees, was used by Du et al. in [68] to perform a game-theoretic analysis 
of Vehicular ad-hoc network security by utilizing the ROA and ROI utility functions. Despite 
sharing the same name with the formalism introduced in [132], the attack-defense tree approach 
used in [68] is built up on defense trees (Section 3.3.2) and does not contain the possibility to 
refine countermeasures. Moreover it does not consider any formal semantics. 

3.3.7 Countermeasure Graphs 

Countermeasure graphs provide a DAG-based structure for identification and prioritization of 
countermeasures. They were introduced by Baca and Petersen [18] in 2010 as an integral part 
of the "countermeasure method for security" which aims at simplifying countermeasure selection 
through cumulative voting. 

To build the graphical model, actors, goals, attacks and countermeasures are identified. Goals 
explain why someone attacks a system, actors are the ones that attack the system, goal explain 
why actors attack a system, attacks detail how the system could get attacked and countermeasures 
describe how attacks could be prevented. When the representing events are related, edges are drawn 
between goals and actors, actors and attack as well as between attacks and countermeasures. More 
specifically, an edge is drawn between a goal and an actor if the actor pursues the goal. An edge 
is inserted between an actor and an attack, if the actor is likely to be able to execute the attack. 
Finally an edge is drawn between an attack and a countermeasure if the countermeasure is able to 
prevent the attack. Priorities are assigned to goals, actors, attacks and countermeasures according 
to the rules of hierarchical cumulative voting [20]. The higher the assigned priority is, the higher 
the threat level of the corresponding event is. 

With the help of hierarchical cumulative voting [20] the most effective countermeasures can be 
deduced. Clever normalization and the fact that countermeasures that prevent several attacks con- 
tribute more to the final result than isolated countermeasures guarantee that the countermeasure 
with the highest computed value is most efficient and should therefore be implemented. 

The methodology is demonstrated on an open source system, a first person shooter called 
Code 43 [18]. 

3.4 Sequential Modeling of Attacks and Defenses 
3.4.1 Insecurity Flows 

In 1997, Moskowitz and Kang described a model called insecurity flows to support risk assess- 
ment [182]. It combines graph theory and discrete probability, offering both graphical represen- 
tation and quantification capabilities to analyze how an "invader can penetrate through security 
holes to various protective security domains". This analysis aims at identifying the most vulnerable 
paths and the most appropriate security measures to eliminate them. 

From a high level perspective, insecurity flows are similar to reliability block diagrams [106] 
used in reliability engineering, without however mentioning such a similarity [182]. The source 
corresponds to the starting point of the attacker, the sink corresponds to the objective of the 
attacker, and the asset under protection. An insecurity flow diagram is a circuit connecting security 
measures, as serial or in parallel, from the sink to the source. Serial nodes must be passed one 
after the other by the attacker, whereas only one out of n connected in parallel must be passed 
to continue its path to the sink. The graph is used to identify insecurity flows and quantify them 
using probabilistic calculations. The paper provides a sound description of the formalism and the 
associated quantifications. 

Based on the circuit, the probability that the insecurity flow can pass through the modeled 
security measures of a given system or architecture can be computed. Probability computation 
formulas for simple serial and parallel patterns are provided, whereas reduction formulas are pro- 
posed for more elaborated circuits (decomposing them into the simple patterns). Several defensive 
architectures can be compared along this metrics. 
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3.4.2 Intrusion DAGs 



Intrusion DAGs (I-DAGs) have been introduced by Wu et al. [292] as the underlying structure for 
attack goals representation in the Adaptive Intrusion Tolerant System, called ADEPTS in 2003. 
The global goal of ADEPTS is to localize and automatically respond to detected, possibly multiple 
and concurrent intrusions on a distributed system. 

I-DAGs are directed acyclic graphs representing intrusion goals in ADEPTS. I-DAGs are not 
necessarily rooted DAGs, i.e., they may have multiple roots. The nodes of an I- DAG represent 
(sub-)goals of an attack and can be associated with an alert from the intrusion detection framework 
described in [293]. A goal represented by a node can only be achieved if (some of) the goals of its 
children are achieved. To model the connection, I-DAGs use standard AND and OR refinement 
features similar to the refinements in attack trees. Each node stores two information sets: a cause 
service set (including all services that may be compromised in order to achieve the goal) and an 
effect service set (including all services that are taken to be compromised once the goal is achieved) . 
The method presented in [292] allows to automatically trigger a response of appropriate severity, 
based on a value which expresses the confidence that the goal corresponding to a node has been 
achieved. This provides dynamic aspects to the ADEPTS methodology. 

Three algorithms have been developed in order to support automated responses to detected 
incidents. The goal of the first algorithm is to classify all nodes as candidates for responses as 
follows. A bottom-up procedure assigns the compromised confidence index to each node situated 
on the paths between the node representing a detected incident and a root node. Then, a value 
called threshold is defined by the user and is used by a top down procedure to label the nodes as 
strong, weak, very weak or non-candidates for potential responses. The second algorithm assigns 
the response index to nodes. The response index is a real number used to determine the response 
to be taken for a given node in the I- DAG. Finally, the third algorithm is based on so called 
effectiveness index. It is responsible for dynamically deciding which responses are to be taken 
next. Intuitively, the effectiveness index of a node is reduced for every detected failure of a response 
action and increased for every successful deployment. 

A lightweight distributed e-commerce system has been deployed to serve as a test bed for the 
ADEPTS tool. The system contained 6 servers and has 26 nodes in the corresponding I-DAG. The 
results of the experiments and analysis are described in [292]. 

In [86] and [291], the authors extend the model of intrusion DAGs to intrusion graphs (I- 
GRAPHs). The main difference is that, contrary to I-DAGs, I-GRAPHs may contain cycles. Nodes 
of an I-GRAPH do not need to be independent. All dependencies between the nodes are depicted 
by the edges between nodes. Additionally to AND and OR refinements, I-GRAPHs also make 
use of quorum edges. A value called minimum required quorum is assigned to quorum edges and 
represents the minimal number of children that need to be achieved in order to achieve the parent 
node. 

3.4.3 Bayesian Defense Graphs 

In a series of papers starting in 2008, Sommestad et al. construct a Bayesian network for security 
(Section 3.2.3) that includes defenses to perform enterprise architecture analysis [76,89,249-251]. 
Their model, explicitly called Bayesian defense graphsm [250], is guided by the idea to depict what 
exists in a system rather than what it is used for [250]. This philosophy was adapted from [113]. 
Bayesian defense graphs are inspired by defense trees (Section 3.3.2) and therefore add countermea- 
sures to Bayesian networks. As a result, the formalism supports a holistic system view including 
attack and defense components. 

Bayesian defense graphs are built up on extended influence diagrams (Section 5.4), including 
utility nodes, decisions nodes chance nodes and arcs. Chance nodes and decision nodes are asso- 
ciated with random variables that may assume one of several predefined and mutually exclusive 
sates. The random variables are given as conditional probability tables (or matrices). Utility nodes 
express the combination of states in chance nodes and decision nodes. Countermeasures, which are 
controllable elements from the perspective of the system owner, are represented as chance nodes 
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with adapted conditional probability tables. Finally, causal arcs (including an AND or OR label) 
are drawn between the nodes indicating how the conditional probabilities are related. A strength 
of Bayesian defense graphs is that they allow to trade-off between collecting as much data as 
possible and the degree of accuracy of the collected data. Through the use of iterative refinement, 
it is possible to reduce the complexity of the model [250]. 

Like all formalisms that involve Bayesian statistics, Bayesian defense graphs use conditional 
probability tables to answer "how do the security mechanisms influence each other?" and "How 
do they contribute to enterprise- wide security?" [249]. The authors of [249] exemplify how to 
compute the expected loss for both the current scenario and potential future scenarios. In [89], 
a suitable subset of a set of 82 security metrics known as Joint Quarterly Readiness Review 
(JQRR) metrics has been selected and adapted to Bayesian Defense graphs. The metrics serve as 
"a posteriori indicators on the historical success rates of hostile attacks" or "indicate the current 
state of countermeasures". The formalism can handle causal and uncertainty measurements at the 
same time, by specifying how to combine the conditional probability tables. 

With the help of a software tool for abstract models [113], Bayesian defense graphs were applied 
by Sommestad et al. to analyze enterprise architectures on numerous occasions. In [76] ongoing 
efforts on Bayesian defense graphs within the EU research project VIKING [272] are summarized. 
The methodology is expanded in three follow-up papers that illustrate security assessment based 
on an enterprise architecture model [249,250] and information flow during a spoofing attack on a 
server [89]. In [251], a real case study was performed with a power distribution operator to assess 
the security of wide-area networks (WANs) used to operate electrical power systems. Since the 
results could not be published the methodology was demonstrated on a fictitious example assessing 
the security of two communication links with the help of conditional probability tables [251]. 

A similar but less developed idea of using random variables, defenses and an inference algorithm 
to compute the expected cost of an attack is presented by Mirembe and Muyeba [174]. 

3.4.4 Security Goal Indicator Trees 

Peine, Jawurek and Mandel devised security goal indicator trees in 2008, in order to support 
security inspections of software development and documents [203] . 

A security goal indicator tree is a tree which combines negative and positive security features 
that can be checked during an inspection, in order to see if a security goal (e.g., secure password 
management) is met. With this objective in mind, "indicators" can be linked in the resulting tree 
structure by three types of relations: Conditional dependencies are represented by a special kind 
of edge, Boolean combination are modeled by OR and AND gates, a "specialization" relation is 
represented by a UML-like inheritance symbol. Moreover, a notion of "polarity" is defined for 
each node, attributing positive or negative effect of a given property on security. The definition of 
security goal indicator trees is semi-formal. 

The formalism does not support quantitative evaluations. 

Security goal indicator trees are implemented in a prototype tool, mentioned in [203] . They are 
used to formalize security inspection processes for a distributed repository of digital cultural data 
in e-tourism application in [115]. The formalism is extended to dependability inspection in [128]. 

3.4.5 Attack-Response Trees 

In 2009, Zonouz, Khurana, Sanders, and Yardley introduced attack-response trees (ART) as a part 
of a methodology called response and recovery engine (RRE), which was proposed to automate 
the intrusion response process. The goal of the RRE is to provide an instantaneous response 
to intrusions and thus eliminate the delay which occurs when the response process is performed 
manually. The approach is modeled as a two-player Stackelberg stochastic game between the leader 
(RRE) and the follower (attacker). Attack- response trees have been used in [300], for the first time. 
This paper constitutes a part of the Ph.D. thesis of Zonouz [299]. 

ARTs are an extension of attack trees (Section 3.1.1) that incorporate possible response actions 
against attacks. They provide a formal way to describe the system security based on possible intru- 
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sion and response scenarios for the attacker and the response engine, respectively. An important 
difference between attack trees and attack-response trees is that the former represent all possible 
ways of achieving an attack goal and the latter are built based on the attack consequences 6 . In an 
attack-response tree, a violation of a security property, e.g., integrity, confidentiality or availability, 
is assigned to the root node (main consequence) . Refining nodes represent sub-consequences whose 
occurrence implies that the parent consequence will take place. Some consequence nodes are then 
tagged by response nodes that represent response actions against the consequence to which they 
are connected. 

The goal of attack-response trees is to probabilistically verify whether the security property 
specified by the root of an attack-response tree has been violated, given the sequence of the received 
alerts and the successfully taken response actions. First, a simple bottom-up procedure is applied 
in the case when values and 1 are assigned to the leaf nodes. More precisely, when a response 
assigned to a node v is activated (i.e., is assigned with 1), the values in the subtree rooted in v are 
reset to 0. Second, [300] also discusses the situation when uncertainties in intrusion detections and 
alert notifications render the determination of Boolean values impossible. In this case, satisfaction 
probabilities are assigned to the nodes of attack-response trees and a game-theoretic algorithm is 
used to decide on the optimal response action. In [301], the RRE has been extended to incorporate 
both IT system-level and business-level metrics to the model. Here, the combined metrics are used 
to recommend optimal response actions to security attacks. 

The RRE has been implemented on top of the intrusion detection system (IDS) Snort 2.7, 
as described in [299]. A validation of the approach on a SCADA system use case [300] and a 
web-based retail company example [301] has shown that this dynamic method performs better 
than static response mechanisms based on lookup tables. The RRE allows to recover the system 
with lower costs and is more helpful than static engines when a large number of IDS alerts from 
different parts of the system are received. 

As pointed out in [225], the approach described in this section suffers from the state space 
explosion problem. To overcome this problem, attack countermeasure trees (Section 3.3.5) have 
been introduced. Their authors propose efficient algorithms for selecting an optimal set of coun- 
termeasures. 

3.4.6 Boolean Logic Driven Markov Process 

Boolean logic driven Markov processes (BDMPs) are a general security modeling formalism, which 
can also complete generic risk assessment procedures. It was invented by Bouissou and Bon in 2003 
in the safety and reliability area [27], and was adapted to security modeling by Pietre-Cambacedes 
and Bouissou in 2010 [208,209] 7 . Its goal is to find a better trade-off between readability, modeling 
power and quantification capabilities with respect to the existing formalisms in general and attack 
trees in particular. 

BDMPs combine the readability of classical attack trees with the modeling power of Markov 
chains. They change the attack tree semantics by augmenting it with links called triggers. In 
a first approach, triggers allow modeling of sequences and simple dependencies by conditionally 
"activating" sub-trees of the global structure. The root (top event) of an BDMP is the objective 
of the attacker. The leaves correspond to attack steps or security events. They are associated to 
Markov processes, dynamically selected in function of the states of some other leaves. They can 
be connected by a wide choice of logical gates, including AND, OR, and PAND gates, commonly 
used in dynamic fault trees (Section 3.2.8). The overall approach allows for sequential modeling in 
an attack tree-like structure, while enabling efficient quantifications. BDMPs for security are well 
formalized [208]. 

Success or realization parameters (mean time to success or to realization) are associated to 
the leaves, depending on the basic event modeled. Defense-centric attributes can also be added, 
reflecting detection and reaction capabilities (the corresponding parameters are the probability 

6 A reader may notice that what the authors of [300] call "sub-consequences" are in fact the causes of the main 
consequence. 

7 The original idea was introduced in a fast abstract by the same authors in 2009 [207] 
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or the mean-time to detection for a given leaf, and the reduction of chance of success in case of 
detection). BDMPs for security allow for different types of quantification. These quantifications 
include the computation of time-domain metrics (overall mean-time to success, probability of 
success in a given time, ordered list of attack sequences leading to the objectives), attack tree 
related metrics like costs of attacks, handling of Boolean indicators (e.g., specific requirements), 
and risk analysis oriented tools like sensibility graphs by attack step or event [211], etc. 

The model construction and its analysis are supported by an industrial tool, called KB3 [72]. 
In [211], implementation issues and user feedback are discussed and analyzed. BDMPs are used 
in [112,210] to integrate safety and security analyses while [138] develops a realistic use-case based 
on the Stuxnet attack. 

In several papers [208,209,211], the authors point out the intrinsic limits of BDMPs to model 
cyclic behaviors and loops, as well as the difficulties to assign relevant values for the leaves. 

3.4.7 Security Goal Models 

In 2010, Security goal models (SGMs) were formalized by Byers and Shahmehri [42] in oder to 
identify the causes of software vulnerabilities and model their dependencies. They were introduced 
as a more expressive replacement for attack trees (Section 3.1.1), security goal indicator trees (Sec- 
tion 3.4.4), vulnerability cause graphs (Section 3.2.7), and security activity graphs (Section 3.3.4). 
The root goal of a SGM corresponds to a vulnerability. "Starting with the root, subgoals are 
incrementally identified until a complete model has been created" [237]. 

In SGMs, a goal can be anything that affects security or some other goal, e.g., it can be 
a vulnerability, a security functionality, a security-related software development activity or an 
attack. SGMs have two types of goal refinements: one type represents dependencies and one type 
modeling information flow. Dependency nodes are connected with solid edges (dependence edge) 
and are depicted by white nodes for contributing subgoals and by black nodes for countering 
subgoals. Information edges are displayed with dashed edges. The formalism consists of a syntactic 
domain (elements that make up the model), an abstract syntax (how elements can be combined), 
a visual representation (used graphical symbols) and a semantic transformation from the syntactic 
domain to the semantic domain. The syntactic domain consists of the root, subgoals (contributing 
or counteracting), dependency edges, operators AND and OR that express the connection of 
dependency edges, annotation connected to nodes by annotation edges, stereotype (usually an 
annotation about a dependency edge), ports that model information flow and information edges 
that connect ports. The abstract syntax defined in a UML class diagram [237]. 

It is possible to evaluate whether a security goal was successfully reached or not. To do this, 
each cause is defined with a logical predicate (true/false). Then the predicates are composed using 
Boolean logic and taking the information from the information edges into account. 

SGM were used in a case study about passive testing vulnerability detection, i.e., examining 
the traces of a software system without the need for specific test inputs. In a four step testing 
procedure vulnerabilities are first modeled using SGMs. In the next step, causes are formally 
defined before SGMs are converted into vulnerability detection conditions (VDC). In the final 
step vulnerabilities are checked based on the VDCs. In [237] this procedure is performed on the 
xine media player [260] where an older version contained the CVE-2009-1274 vulnerability. The 
case study is executed with the help of "Testlnv-Code", a program developed by Montimage that 
can handle VCDs. 

In [42], the authors explicitly state that they have defined transformations to and from attack 
trees VCGs, SAGs and SGITs so that SGMs can be used with possibly familiar notation. (The 
transformations, however, were omitted due to space restrictions.) 
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3.4.8 Unified Parameterizable Attack Trees 

In 2011, Wang, Whitley, Phan and Parish introduce unified parameterizable attack trees 8 [278]. 
As the name suggests, the formalism was created as a foundation to unify numerous existing 
extensions of attack trees (Section 3.1.1). The formalism generalizes the notions of connector types , 
edge augmentations and (node) attributes. With the help of these generalizations it is possible 
to describe other extensions of attack trees as structural extensions, computation extensions or 
hybrid extensions. 

Unified parameterizable attack trees are defined as a 5-tuple, consisting of a set of nodes, a 
set of edges, a set of allowed connectors (O-AND i.e., a time or priority based AND, U-AND i.e., 
an AND with a threshold condition and OR), a set of attributes and a set of edge augmentation 
structures that allows to specify edge labels. Using this definition, the authors of [278] identify 
defense trees (Section 3.3.2), attack countermeasure trees (Section 3.3.5), attack-response trees 
(Section 3.4.5), attack-defense trees (Section 3.3.6), protection trees (Section 3.3.3), OWA trees 
(Section 3.1.4), and augmented attack trees (Section 3.1.3) as structure-based extensions of attack 
tree that are covered by unified parameterizable attack trees. They classify multi-parameter attack 
trees (Section 3.1.5 and 3.2.9) as a computational extension of attack trees. 

The formalism classifies attributes into the categories of "attack accomplishment attributes", 
"attack evaluation attributes" and "victim system attributes", but does not specify how to perform 
quantitative evaluations. 

Unified parameterizable attack trees are primarily built upon augmented attack trees (Sec- 
tion 3.1.3). In fact, the authors indicate how to instantiate the node attributes, the edge augmen- 
tation and the connector type to obtain an augmented attack tree. 

4 Summary of the surveyed formalisms 

In this section, we provide a consolidated view of all formalisms introduced in Section 3. Tables 2— 
4 characterize the described methodologies (ordered alphabetically) according to the 13 aspects 
presented in Table 1. The aspects are grouped into formalism features and capabilities (Table 2), 
formalism characteristics (Table 3), and formalism maturity and usability factors (Table 4). This 
tabular view allows the reader to compare the features of the formalisms more easily, it stresses 
their similarities and differences. Furthermore, the tables support a user in selecting the most 
appropriate formalism(s) with respect to specific modeling needs and requirements. We illustrate 
such a support on two exemplary situations. 

Example 1 Let us assume that during a risk assessment, analysts want to investigate and com- 
pare the efficiency of different defensive measures and controls, with respect to several attack 
scenarios. Thereto, they need quantitative elements to support the analysis technique they will 
choose. Furthermore, a software tool and pre-existing use cases are required to facilitate their 
work. Using the corresponding columns from Tables 2-4 (i.e., attack or defensive, quantification, 
tool availability, case study) and choosing the formalisms characterized by appropriate values 
(respectively: both, versatile or specific, industrial or prototype, real(istic)), would help the ana- 
lysts to pre-selected attack countermeasure trees, attack-defense trees, BDMPs, intrusion DAGs, 
and security activity graphs, as potential modeling and analysis techniques. The most suitable 
methodology could then be selected based on more detailed information provided in Section 3. For 
instance, let us assume that the analysis requires the use of measures for probability of success, 
the attacker's costs and the attacker's skills. Checking descriptions of the pre-selected formalisms, 
given in Section 3, would convince the analysts that security activity graphs and intrusion DAGs 
would not allow them to compute the expected quantitative elements. Therefore it would reduce 
the choice to attack countermeasure trees, attack-defense trees, and BDMPs. A more thorough 

8 Wang et al. use British English, thus originally, the name of their formalism is unified parametrizable attack 
trees. 
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investigation of the computational procedures and algorithms described in the referred papers 
would help the analysts to make the final decision on the formalism that best fits their needs. 

Example 2 Now, let us assume that a team of penetration testers wants to illustrate which 
attack paths they have used to compromise different systems. Initially, this does not significantly 
reduce the choice of possible formalisms, since they could use all attack-oriented and all attack 
and defense oriented approaches. However, to keep the model as simple as possible, they start 
the selection process by looking at the attack-oriented methodologies only. Let us assume further 
that the penetration testers also do not need to represent sequences of actions. With a similar 
reasoning as before, they first investigate a possibility of using a static modeling technique. The 
team does not foresee to perform any quantitative analysis. An important requirement, however, is 
to employ a methodology which is already broadly used, with at least rudimentary documented use 
cases on which they could rely to build their own models. Using relevant columns from Tables 2-4 
(i.e., attack or defense, sequential or static, paper count, use cases, quantification) and selecting 
formalisms characterized with appropriate values (respectively: attack or both, sequential or static, 
> 4, real(istic) or toy, versatile, specific or no), the team obtains a large number of applicable 
formalisms. In order to keep the formalism as simple as possible, the analysts decide to narrow 
the set of values they are interested in to: attack, static, > 4, real(istic) or toy, no. This strategy 
yields the following most suitable formalisms: attack trees, augmented attack trees, and parallel 
model for multi-parameter attack trees. The team would then be able to make a final choice of the 
methodology based on complementary investigations starting from the information and references 
provided by the corresponding textual descriptions from Section 3. 



Table 2: Aspects relating to the formalism's modeling capabilities 



Name of formalism 


Attack or 
defense 


Sequential 
or static 


Quantifi- 
cation 


Main 
Purpose 


Extension 


Anti-models 
(Section 3.3.1) 


Both 


Static 


No 


Req. eng. 


New formalism 


Attack countermeasure 
trees (Section 3.3.5) 


Both 


Static 


Specific 


Sec. mod. 


Structural 
Computational 


Attack-defense trees 
(Section 3.3.6) 


Both 


Static 


Versatile 


Sec. mod. 


Structural 
Computational 


Attack-response trees 
(Section 3.4.5) 


Both 


Sequential 


Specific 


Int. det. 


Structural 
Quantitative 


Attack trees 
(Section 3.1.1) 


Attack 


Static 


Versatile 


Sec. mod. 


New formalism 


Augmented attack trees 
(Section 3.1.3) 


Attack 


Static 


Specific 


Sec. mod. 


Structural 
Computational 


Augmented vulnerabil- 
ity trees (Section 3.1.2) 


Attack 


Static 


Specific 


Risk 


Quantitative 


Bayesian attack graphs 
(Section 3.2.4) 


Attack 


Sequential 


Specific 


Risk 


Structural 
Computational 


Bayesian defense graphs 
(Section 3.4.3) 


Both 


Sequential 


Specific 


Risk 


Structural 
Computational 


Bayesian networks for 
security (Section 3.2.3) 


Attack 


Sequential 


Specific 


Risk 


Structural 
Computational 


BDMPs (Section 3.4.6) 


Both 


Sequential 


Versatile 


Sec. mod. 


Order 
Time 


Compromise graphs 
(Section 3.2.5) 


Attack 


Sequential 


Specific 


Risk 


New formalism 



28 



Countcrmeasure graphs 
(Section 3.3.7) 


Both 


Static 


Specific 


Sec. mod. 


Structural 
Computational 


Cryptographic DAGs 
(Section 3.2.1) 


Attack 


Sequential 


No 


Risk 


New formalism 


Defense trees 
(Section 3.3.2) 


Both 


Static 


Specific 


Sec. mod. 


Structural 
Computational 


Dynamic fault trees for 
security (Section 3.2.8) 


Attack 


Sequential 


No 


Sec. mod. 


Order 
Time 


Enhanced attack trees 
(Section 3.2.6) 


Attack 


Sequential 


Specific 


Int. det. 


Order 
Time 


Extended fault trees 
(Section 3.1.6) 


Attack 


Static 


Specific 


Unification 


Structural 


Fault trees for security 
(Section 3.2.2) 


Attack 


Sequential 


No 


Sec. mod. 


Order 


Improved attack trees 
(Section 3.2.10) 


Attack 


Sequential 


Specific 


Risk 


Structural 
Computational 


Insecurity flows 
(Section 3.4.1) 


Both 


Sequential 


Specific 


Risk 


New formalism 


Intrusion DAGs 
(Section 3.4.2) 


Both 


Sequential 


Specific 


Int. det. 


Structural 
Computational 


OWA trees 
(Section 3.1. 4) 


Attack 


Static 


Specific 


Quantitative 


Structural 
Computational 


Parallel model for 
multi-parameter attack 
trees (Section 3.1.5) 


Attack 


Static 


Specific 


Quantitative 


Quantitative 
Computational 


Protection trees 
(Section 3.3.3) 


Defense 


Static 


Specific 


Sec. mod. 


New formalism 


Security activity graphs 
(Section 3.3.4) 


Both 


Static 


Specific 


Soft. dev. 


New formalism 


Security goal indicator 
trees (Section 3.4.4) 


Defense 


Sequential 


No 


Soft. dev. 


New formalism 


Security goal models 
(Section 3.4.7) 


Both 


Sequential 


Specific 


Unification 


Structural 
Computational 


Serial model for multi- 
parameter attack trees 
(Section 3.2.9) 


Attack 


Sequential 


Specific 


Quantitative 


Computational 
Order 


Unified parameterizable 
attack trees 
(Section 3.4.8) 


Both 


Sequential 


Versatile 


Unification 


Structural 


Vulnerability cause 
graphs (Section 3.2.7) 


Attack 


Sequential 


Specific 


Soft. dev. 


Structural 
Order 



Table 3: Aspects relating to the formalism's characteristics 



Name of formalism 


Structure 


Connectors 


Formalization 


Anti-models 
(Section 3.3.1) 


Tree 


AND, OR 


Semi-formal 


Attack countermeasure 
trees (Section 3.3.5) 


Tree 


AND. OR, fc-out-of-n, counter leaves 


Formal 


Attack-defense trees 
(Section 3.3.6) 


Tree 


AND, OR, countermeasures 


Formal 
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Attack-response trees 
(Section 3.4.5) 


Tree 


AND, OR, responses 


Formal 


Attack trees 
(Section 3.1.1) 


Tree 


AND, OR 


Formal 


Augmented attack trees 
(Section 3.1.3) 


Tree 


AND, OR 


Formal 


Augmented vulnerabil- 
lty trees (Section 3.1.2) 


Tree 


AND, OR 


Informal 


Bayesian attack graphs 
(Section 3.2.4) 


DAG 


AND, OR, conditional 
probabilities 


Formal 


Bayesian defense graphs 
(Section 3.4.3) 


DAG 


AND, OR, conditional 
probabilities 


Formal 


Bayesian networks for 
security (Section 3.2.3) 


DAG 


AND, OR, conditional 
probabilities 


Formal 


BDMPs (Section 3.2.3) 


DAG 


AND, OR, PAND, approx. OR, triggers 


Formal 


Compromise graphs 
(Section 3.2.5) 


Unspecified 


None 


Formal 


Countermeasure graphs 
(Section 3.3.7) 


DAG 


Countermeasures 


Informal 


Cryptographic DACs 
(Section 3.2.1) 


DAG 


Dependence edges 


Informal 


Defense trees 
(Section 3.3.2) 


Tree 


AND, OR, counter leaves 


Semi-formal 


Dynamic fault trees for 
security (Section 3.2.8) 


Tree 


AND, OR, PAND, SEQ, FDEP, CSP 


Informal 


Enhanced attack trees 
(Section 3.2.6) 


Tree 


AND, OR, ordered- AND 


Formal 


Extended fault trees 
(Section 3.1.6) 


Tree 


AND, OR, merge gates 


Formal 


Fault trees for security 
(Section 3.2.2) 


Tree 


AND, OR, PAND, XOR, inhibit 


Informal 


Improved attack trees 
(Section 3.2.10) 


Tree 


AND, OR, sequential AND 


Informal 


Insecurity flows 
(Section 3.4.1) 


Unspecified 


None 


Formal 


Intrusion DAGs 
(Section 3.4.2) 


DAG 


AND, OR 


Semi-formal 


OWA trees 
(Section 3.1.4) 


Tree 


OWA operators 


Formal 


Parallel model for 








multi-parameter attack 


Tree 


A TVT1 — V / \ I "\ 

AND, OR 


Formal 


trees (Section 3.1.5) 








Protection trees 
(Section 3.3.3) 


Tree 


AND, OR 


Informal 


Security activity graphs 
(Section 3.3.4) 


DAG 


AND, OR, split gate 


Semi-formal 


Security goal indicator 
trees (Section 3.4.4) 


Tree 


AND, OR, dependence 
edge, specialization edge 


Semi-formal 


Security goal models 
(Section 3.4.7) 


DAG 


AND, OR, dependence edge, informa- 
tion edge 


Formal 
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Serial model for multi- 
parameter attack trees 
(Section 3.2.9) 


Tree 


AND, OR, ordered leaves 


Formal 


Unified parameterizable 
attack trees 
(Section 3.4.8) 


Tree 


AND, OR, PAND, time-based AND, 
threshold AND 


Formal 


Vulnerability cause 
graphs (Section 3.2.7) 


DAG 


AND, OR, sequential AND 


Informal 



Table 4: Aspects related to the formalism's maturity and usability 



Name of formalism 


Tool 

availability 


Case study 


External use 


Paper 
count 


Year 


Anti-models 
(Section 3.3.1) 


No 


No 


No 


3 


2006 


Attack countermeasure 
trees (Section 3.3.5) 


Prototype 


Real(istic) 


No 


4 


2010 


Attack-defense trees 
(Section 3.3.6) 


Prototype 


Real(istic) 


Collaboration 


6 


2010 


Attack-response trees 
(Section 3.4.5) 


Prototype 


Toy case study 


No 


3 


2009 


Attack trees 
(Section 3.1.1) 


Commercial 


Real(istic) 


Independent 


> 100 


1991 


Augmented attack trees 
(Section 3.1.3) 


No 


Real(istic) 


Independent 


6 


2005 


Augmented vulnerabil- 
ity trees (Section 3.1.2) 


No 


Real(istic) 


Independent 


3 


2003 


Bayesian attack graphs 
(Section 3.2.4) 


Commercial 


Toy case study 


Independent 


10 


2005 


Bayesian defense graphs 
(Section 3.4.3) 


Prototype 


Real(istic) 


No 


5 


2008 


Bayesian networks for 
security (Section 3.2.3) 


Commercial 


Real(istic) 


Independent 


14 


2004 


BDMPs (Section 3.4.6) 


Commercial 


Real(istic) 


Independent 


5 


2010 


Compromise graphs 
(Section 3.2.5) 


No 


Real(istic) 


Collaboration 


3 


2006 


Countermeasure graphs 
(Section 3.3.7) 


No 


Toy case study 


No 


1 


2010 


Cryptographic DAGs 
(Section 3.2.1) 


No 


No 


No 


1 


1996 


Defense trees 
(Section 3.3.2) 


No 


No 


No 


3 


2006 


Dynamic fault trees for 
security (Section 3.2.8) 


No 


No 


No 


1 


2009 


Enhanced attack trees 
(Section 3.2.6) 


No 


No 


No 


1 


2007 


Extended fault trees 
(Section 3.1.6) 


No 


No 


No 


1 


2007 


Fault trees for security 
(Section 3.2.2) 


Commercial 


Real(istic) 


Independent 


3 


2003 
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Improved attack trees 
(Section 3.2.10) 


No 


No 


No 


1 


2011 


Insecurity flows 
(Section 3.4.1) 


No 


No 


No 


1 


1997 


Intrusion DAGs 
(Section 3.4.2) 


Prototype 


Real(istic) 


No 


2 


2003 


OWA trees 
(Section 3.1.4) 


No 


No 


No 


2 


2005 


Parallel model for 












multi-parameter attack 


Prototype 


Real(istic) 


Collaboration 


5 


2006 


trees (Section 3.1.5) 












Protection trees 
(Section 3.3.3) 


No 


Toy case study 


No 


4 


2006 


Security activity graphs 
(Section 3.3.4) 


Prototype 


Real(istic) 


No 


2 


2006 


Security goal indicator 
trees (Section 3.4.4) 


Prototype 


Real(istic) 


No 


3 


2008 


Security goal models 
(Section 3.4.7) 


No 


Real(istic) 


No 


2 


2010 


Serial model for multi- 












parameter attack trees 


Prototype 


No 


No 


3 


2010 


(Section 3.2.9) 












Unified parameterizable 












attack trees 


No 


No 


No 


1 


2011 


(Section 3.4.8) 












Vulnerability cause 
graphs (Section 3.2.7) 


Commercial 


Real(istic) 


Independent 


4 


2006 



5 Alternative Methodologies 

We close this survey with a short overview of alternative methodologies for security modeling and 
analysis. The formalisms described here are outside the main scope of this paper, because they 
were not originally introduced for the purposes of attack and defense modeling or they are not 
based on the DAG structure. However, for the sake of completeness, we find important to briefly 
present those approaches as well. The objective of this section is to give pointers to other existing 
methodological tools for security assessment, rather than to perform a thorough overview of all 
related formalisms. This explains why the description of the formalisms given here is less complete 
and structured than the information provided in Section 3. 

5.1 Petri Nets for Security 

In the mid 1990s, models based on Petri nets have been applied for security analysis [52,139]. 
In 1994, Kumar and Spafford [139] adopted colored Petri nets for security modeling. They illustrate 
how to model reference scenarios for an intrusion detection device. Also in 1994, Dacier [52] used 
Petri nets in his Ph.D. thesis as part of a larger quantification model that describes the progress of 
an attacker taking over a system. A useful property of Petri nets is their great modeling capability 
and in particular their ability to take into account the sequential aspect of attacks, the modeling 
of concurrent action and various forms of dependency. Petri nets are widely used and have various 
specific extensions. To corroborate this statement, we list a few existing ones. Kumar and Spafford's 
work relies on colored Petri nets [139], Dacier's on stochastic Petri nets [52], McDermott's on 
disjunctive Petri nets [162], Horvath and Dorges' on reference nets [101], Dalton II et al.'s on 
generalized stochastic Petri nets [57], Pudar et al.'s on deterministic time transition Petri nets [216] 
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and Xu and Nygard's on aspect- oriented Petri nets [295]. Several articles on Petri nets merge the 
formalism with other approaches. Horvath and Dorges combine Petri nets with the concept of 
security patterns [101] while Dalton II et al. [57], and more thoroughly Pudar et al. [216], combine 
Petri nets and attack trees. 

In 1994, Dacier embedded Petri nets into a higher level formalism called privilege graphs. They 
model an attacker's progress in obtaining access rights for a desired target [52, 53]. In a privilege 
graph, a node represents a set of privileges and an edge a method for transferring these privileges 
to the attacker. This corresponds to the exploitation of a vulnerability. The model includes an 
attacker's "memory" which forbids him to go through privilege states that he has already acquired. 
In addition, an attacker's "good sense" is modeled which prevents him from regressing. In [54], 
Dacier et al. proposed to transform a privilege graph into Markov chain corresponding to all 
possible successful attack scenarios. The method has been applied to help system administrators 
to monitor the security of their systems. 

In [297], Zakzewska and Ferragut presented a model extending Petri nets in order to model 
real-time cyber conflicts. This formalism is able to represent situational awareness, concurrent ac- 
tions, incomplete information and objective functions. Since it makes use of stochastic transitions, 
it is well suited to reason about stochastic non-controlled events. The formalism is used to run 
simulations of cyber attacks in order to experimentally analyze cyber conflicts. The authors also 
performed a comparison of their extended Petri nets model with other security modeling tech- 
niques. In particular, they showed that extended Petri nets are more readable and more expressive 
than attack graphs, especially with respect to the completeness of the models. 

5.2 Attack Graphs 

The term attack graph has been first introduced by Phillips and Swiler [205, 256] in 1998, and 
has extensively been used ever since. The nodes of an attack graph represent possible states of a 
system during the attack. The edges correspond to changes of states due to an attacker's actions. 
An attack graph is generated automatically based on three types of inputs: attack templates 
(generic representations of attacks including required conditions), a detailed description of the 
system to be attacked (topology, configurations of components, etc.), and the attacker's profile 
(his capability, his tools, etc.). Quantifications, such as average probabilities or time to success, 
can be deduced by giving weight to the edges and by finding shortest paths in the graph. 

Starting in 2002, Sheyner [238,239] made extensive contributions to popularize attack graphs 
by associating them with model checking techniques. To limit the risk of combinatorial explosion, 
a large number of methods were developed. Ammann et al. [8] restricted the graphs by exploiting 
a monotony property, thereby eliminating backtracking in terms of privilege escalation. Noel and 
Jajodia [110, 191] took configuration aspects into account. A complete state of the art concerning 
the contributions to the field between 2002 and 2005 can be found in [150]. In 2006, Wang et 
al. introduced a relational model for attack graphs [283]. The approach facilitates interactive 
analysis of the models and improves its performance. Ou et al. [197] optimized the generation 
and representation of attack graphs by transforming them into logical attack graphs of polynomial 
size with respect to the number of components of the computer network analyzed. During the 
same year, Ingols et al. [105] proposed multiple-prerequisite graphs, which also severely reduce the 
complexity of the graphs. In [170], Mehta et al. proposed an algorithm for classification of states 
in order to identify the most relevant parts of an attack graph. In 2008, Malhotra et al. [153] 
did the same based on the notion of attack surface described in [155]. The vast majority of the 
authors mentioned have also worked on visualization aspects [100, 189, 190, 290]. Kotenko and 
Stepashkin [137] described a complete software platform for implementing concepts and metrics of 
attack graphs. On a theoretical level, Braynov and Jadliwala [28] extended the model with several 
attackers. 

Starting in 2003, the problem of quantitative assessment of the security of networked systems 
using attack graphs has been extensively studied [191,279-282]. The work presented in [191] 
and [280] focuses on minimal cost of removing vulnerabilities in hardening a network. In [281], 
the authors introduced a metric, called attack resistance, which is used to compare the security of 
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different network configurations. The approach was then extended in [282] into a general abstract 
framework for measuring various aspects of network security. In [279], Wang et al. introduced a 
metric incorporating probabilities of the existence of the vulnerabilities considered in the graph. 

In his master thesis, Louthan IV [109] proposed extensions to the attack graph modeling 
framework to permit modeling continuous, in addition to discrete, system elements and their in- 
teractions. In [279], Wang et al. addressed the problem of likelihood quantification of potential 
multi-step attacks on networked environments, that combine multiple vulnerabilities. They de- 
veloped an attack graph-based probabilistic metric for network security and proposed heuristics 
for efficient computation. In [192], Noel et al. used attack graphs to understand how different 
vulnerabilities can be combined to form an attack on a network. They simulated incremental net- 
work penetration and assessed the overall security of a network system by propagating attack 
likelihoods. The method allows to give scores to risk mitigation options in terms of maximizing 
security and minimizing cost. It can be used to study cost/benefit trade-offs for analyzing return 
on security investment. 

Dawkins and Hale [62] developed a concept similar to attack graphs called attack chains. The 
model is based on a deductive tree structure approach but also allows for inductive reasoning 
using goal-inducing attack chains, to extract scenarios leading to a given aim. These models are 
also capable of generating attack trees, which may be quantified by conventional methods. Aspects 
concerning software implementation are described in [49]. 

5.3 Approaches Derived from UML Diagrams 

We start this section with a short description of two formalisms derived from UML diagrams, 
namely abuse cases of McDermott and Fox [163] and misuse cases of Sindre and Opdahl [5,244-247] 
which were later extended by R0stad in [222]. These techniques are not specifically intended to 
model attacks but rather to capture threats and abusive behavior which have to be taken into 
account when eliciting security requirements (for misuse cases) as well as for design and testing 
(for abuse cases). The flexibility of misuse and abuse cases allows for expressive graphical modeling 
of attack scenarios without mathematical formalization that supports quantification. 

In [85], Firesmith argues that misuse and abuse cases are "highly effective ways of analyzing 
security threats but are inappropriate for the analysis and specification of security requirements". 
The reasoning is that misuse cases focus on how misusers can successfully attack the system. 
Thus they often model specific architectural mechanisms and solutions, e.g., the use of passwords, 
rather than actual security requirements, e.g., authentication mechanisms. To specify security 
requirements, he suggested to use security use cases. Security use cases focus on how an application 
achieves its goals. According to Firesmith, they provide "a highly-reusable way of organizing, 
analyzing, and specifying security requirements" [85]. 

Diallo et al. presented a comparative evaluation of the common criteria [1], misuse cases, and 
attack trees [67]. Opdahl and Sindre [194] compared usability aspects and modeling features of 
misuse cases and attack trees. UML-based approaches can be combined with other types of models. 
The combination of misuse cases and attack trees appears not only to be simple but also useful 
and relevant [172,262]. In [120], Karpati et al. adapted use case maps to security as misuse case 
maps. Katta et al. [123] combined UML sequence diagrams with misuse cases in a new formalism 
called misuse sequence diagrams. A misuse sequence diagram represents a sequence of attacker 
interactions with system components and depicts how the components were misused over time by 
exploiting their vulnerabilities. The authors of [123] performed usability and performance com- 
parison of misuse sequence diagrams and misuse case maps. In [204], Karpati et al. integrated 
five different representation techniques in a method called hacker attack representation method 
(HARM). The methodologies used in HARM are: attack sequence descriptions (summarizing at- 
tacks in natural language), misuse case maps (depicting the system architecture targeted by the 
attack and visualizing the traces of the exploits), misuse case diagrams (showing threats in re- 
lation to the wanted functionality) attack trees (representing the hierarchical relation between 
attacks) and attack patterns (describing an attack in detail by adding information about context 
and solutions). Combining such diverse representation techniques has two goals. First, it provides 
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"an integrated view of security attacks and system architecture". Second, the HARM method is 
especially well suited in involving different stakeholders, including non-technical people preferring 
informal representations. 

In [243], Sindre adapted UML activity diagrams to security. The resulting mal-activity diagrams 
constitute an alternative for misuse cases when the author considers the latter to be unsuitable. 
This is for instance the case in situations where a large numbers of interactions need to be specified 
within a or outside a system. Case studies mainly concern social engineering attacks [121]. 

5.4 Isolated models 

In this section we gather a number of isolated models. Most of them contain cycles and therefore 
are outside of the main scope of this paper. However, we mention them because they build upon 
a formalism described in Section 3. 

The stratified node topology was proposed by Daley et al. [55] as an extension of attack trees, 
in 2002. The formalism consists of a directed graph which is aimed at providing a context sen- 
sitive attack modeling framework. It supports incident correlation, analysis and prediction and 
extends attack trees by separating the nodes into three distinct classes based on their function- 
ality: event-level nodes, state-level nodes and top-level nodes. The directed edges between the 
nodes are classified into implicit and explicit links. Implicit links allow individual nodes to imply 
other nodes in the tree; explicit links are created when an attack provides a capability to execute 
additional nodes, but does not actually invoke a new instance of a node. As in attack trees, the set 
of linked nodes can be connected disjunctively as well as conjunctively. In comparison with attack 
trees, the authors drop the requirement of a designated root node, along with the requirement 
that the graphs have to be acyclic. Due to the functional distinction of the nodes, the stratified 
node topology can keep the vertical ordering, even if the modeled scenario is cyclic. 

In 2010, Abdulla et al. [2] described a model called attack jungles. When trying to use attack 
trees as formalized by Mauw and Oostdijk in [161] to illustrate the security of a GSM radio 
network, the authors of [2] encountered modeling problems related to the presence of cycles as well 
as analysis problems related to reusability of nodes in real life scenarios. This led them to propose 
attack jungles, which extend attack trees with multiple roots, reusable nodes and cycles that allow 
for modeling of attacks which depend on each other. Attack jungles are formalized as multigraphs 
and their formal semantics extend the semantics based on multisets proposed in [161]. In order to 
find possible ways of attacking a system, a backwards reachability algorithm for analysis of attack 
jungles was described. Moreover, the notion of an attribute domain for quantitative analysis, as 
proposed for attack trees in [161], is extended to fit the new structure of attack jungles. By dividing 
attack components (nodes) into reusable and not reusable ones, it is possible to reason about and 
better analyze realistic scenarios. For instance, in an attack jungle a component used once can be 
reused multiple times without inducing any extra cost. 

Extended influence diagrams [114] form another related formalism which is not based on a DAG 
structure. Extended influence diagrams are built upon influence diagrams, introduced by Matheson 
and Howard in the 1960s [160], which, in turn, are an extension of Bayesian networks. Influence 
diagrams are also known as relevance diagrams, decision diagrams or decision networks and are 
used to provide a high-level visualization of decision problems under uncertainty [82]. Extended 
influence diagrams allow to model the relationships between decisions, events and outcomes of an 
enterprise architecture. They employ the following three types of node: ellipses which represent 
events (also known as chance nodes), rectangles which depict decision nodes and diamonds which 
represent utility nodes (or outcomes). In addition the formalism allows to specify how a node 
is defined, how well it can be controlled and how the nodes relate to each other. The latter is 
achieved with different types of edges. Moreover, transformation rules between graphs govern 
switching between different levels of abstraction of a scenario (expanding and collapsing). The 
rules also ensure that graphs do not contradict each other. In [140], the authors show how to elicit 
knowledge from scientific texts, generating extended influence diagrams and in [76] the authors 
outline how extended influence diagrams can be used for cyber security management. 
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6 Conclusion 



This work presents a complete and methodical overview of DAG-based techniques for modeling at- 
tack and defense scenarios. Some of the described methodologies have extensively been studied and 
are widely used to support security and risk assessment processes. Others emerged from specific, 
practical developments and have remained isolated methods. This survey provides a structured 
description of the existing formalisms, gives pointers to related papers, tools and projects, and 
proposes a general classification of the presented approaches. To classify the formalisms, we have 
used 13 aspects concerning graphical, formal and usability characteristics of the analyzed models. 

Two general trends can be observed in the field of graphical security modeling: unification 
and specification. The objective of the methodologies developed within the first trend is to unify 
existing approaches and propose general solutions that can be used for analysis of a broad spectrum 
of security scenarios. The corresponding formalisms are well suited for reasoning about situations 
involving diversified aspects, such as digital, physical and social components, simultaneously. Such 
models usually have sound formal foundations and are extensively studied from a theoretical 
point of view. They are augmented with formal semantics and a general mathematical framework 
for quantitative analysis. Examples of such models developed within the unification trend are 
attack-defense trees, unified parameterizable attack trees, multi-parameter attack trees, OWA 
trees, Bayesian attack graphs, and Bayesian defense graphs. 

The second observed trend, i.e., the specification trend, aims at developing methodologies 
for addressing domain specific security problems. Studied domains include intrusion detection 
(e.g., attack-response trees, intrusion DAGs), secure software development (e.g., security activity 
graphs, security goal indicator trees), and security requirements engineering (e.g., anti-models). 
Formalisms developed within this trend are often based on empirical studies and practical needs. 
They concentrate on domain specific metrics, such as the response index, which is used for the 
analysis of intrusion DAGs. These approaches often remain isolated and seldom relate to or build 
upon other existing approaches. 

The multitude of methodologies presented in this survey shows that graphical security modeling 
is a young but very rapidly growing area. Thus, further development is necessary and new directions 
need to be explored before security assessment can fully benefit from graphical models. One of 
the research questions which has not yet received enough attention is building graphical models 
from pre-existing attack templates and patterns. Addressing this question would make automatic 
model creation possible and replace the tedious, error-prone, manual construction process. It would 
therefore strongly relieve the industrial sector when building large-scale practical models. 

The idea of reusing attack patterns is not new. It has already been mentioned in 2001 by Moore 
et al. [178]. An excellent initiative was taken by the FP7 project SHIELDS [241], in which the 
Security Vulnerability Repository Service (SVRS) has been developed. The SVRS is an on-line 
library of various security models including attack trees [242]. A natural follow-up step would 
be to propose methods for automatic or semi-automatic construction of complex, specific models 
from general attack or vulnerability patterns. This would require developing algorithms for correct 
composition and comparison of models, standardizing employed node labels and introducing an 
agent-based view into the formalisms. 

Using security patterns makes threat analysis more efficient and accurate. First generating a 
general model from existing libraries constitutes a good starting point for further model refinement 
and analysis. Furthermore, although new technological opportunities arise every day, empirical 
studies show that most attackers reuse the same attack vectors with little or no modification. 
Often the same company is attacked several times by an intruder exploiting the same already 
known vulnerability. 

There still exists a gap between theoretical research and practical employment of graphical 
security models. Tighter interaction between the scientific and industrial security communities 
would be very beneficial for the future of the field. Setting up dedicated events, such as work- 
shops, conferences or panel discussions, would provide a platform for the exchange of ideas, closer 
collaboration and a faster dissemination of results. This would allow practitioners to better under- 
stand the capabilities of theoretical models and give scientists an opportunity to learn what the 
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practical and industrial needs are. Once a bridge between the two communities is built, a natural 
next step will be to include graphical models into standardized and commonly used auditing and 
risk assessment tools and practices. Due to the sound formal foundations of the graphical mod- 
els as well as their user friendliness, this would greatly improve the quality and usability of the 
currently used, mostly table-based, practical risk and auditing methodologies. 
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